Project description

develop and maintain scalable data pipelines
create efficient Extract, Transform, Load (ETL) processes to handle data transformation and preparation
ensure data accuracy and integrity across various sources
work with data warehousing technologies to organize and store large volumes of data
optimize data storage and retrieval for analytical and operational purposes
collaborate with data scientists to deploy machine learning models (incl. LLMs) into production environments. Leverage tools like Kubeflow, MLflow to streamline the end-to-end ML lifecycle
to ensure seamless integration of ML models with data pipelines
collaborate with business teams to enhance data models that feed business intelligence tools
foster data-driven decision-making by making data accessible to relevant stakeholders
quality assurance and consistency
implement software and methodologies for data correction, reconciliation, and quality checking
maintain data consistency across different data sources

Project requirements

advanced Python knowledge for data processing and scripting
hands-on experience with one or more cloud services (Azure, AWS, GCP)
proficiency in SQL and NoSQL databases
experience with Data as Code ; version control, small and regular commits, unit tests, CI/CD, packaging, familiarity with containerization tools such as Docker and Kubernetes
experience with Infrastructure as Code is a plus
strong problem-solving skills and attention to detail
good communication skills, fluent English

Are you interested in this project?

Collapse project information

New to the world of IT freelancing ?

Freedom, flexibility, greater control over finances and career. Freelancing has evolved and offers much more today. See what's in store for you and how it will change your life.