And they’re leaving it in my hands 🙃
Anyone have any suggestions for tools/technologies to look into? It will most likely take the form of csv extracts from system A with fields/objects to be mapped to system B.
Also, this question is truly an instance of how the internet is a fucking wasteland. I could’ve asked this question 5 or 6 years ago on Reddit and gotten something meaningful, but today I would get replies from tons of company accounts trying to sell me their garbage solutions based on LLM wrappers
Without know specifics kinda hard to know what to recommend but you’re probably going to need a basic structure of properly backing up your dumps from system A, build a python script to convert the schema of system A to the schema of system B and properly store a backup of your new files to be uploaded to system B. Depends on how complicated you wanna make the process.
If you’re starting from scratch, you’re basically the trailblazer here. Is your company giving you a budget or a team to build the workflow?
Use
dbt, host it on whatever they got (vSphere, cloud VMs, w/e) or schedule it the same way their “Devops” or “CICD” pipelines are orchestrated. Makes it easier if you wind up having a bunch of workers move data in parallel during the migration window or if you need to do a rolling refresh of the destination for validation or something.If you can get access to the source database, query the
INFORMATION_SCHEMAviews (select * from information_schema.columns) to get the data types of the fields in the source system. Same for the destination it sounds like, but once you have the columns and tables present in both systems you can probably start mapping it easily or chatbots probably do OK at it.What is dbt? Dealing with azure here too, but no access to anything other than SOQL (platform sits on top of salesforce)
ctrl+c then ctrl+v
🧠
ctrl+x then ctrl-v it’ll be fine
I would start by documenting the schemas (data types and optional fields). Then use shell scripts to transform or join any data to be ingested to another system. Not sure what you’re working with so it’s hard to give any more concrete advice.
It’s a platform that sits on top of salesforce
Bleh hate it lol would be so sick if I could query SQL information schema tables
rclone and rsync maybe?




