Senior Data Engineer Python
SwedenCompetitive0 applicants
About this role
Job Description:
The client is seeking an experienced Senior Data Engineer with strong expertise in Python to support advanced data engineering initiatives within an automotive environment. This role involves designing and maintaining robust data pipelines, working with large-scale test data, and contributing to distributed system architecture. The ideal candidate will have a strong background in enterprise-level development and hands-on experience with ETL processes and cloud infrastructure.
Responsibilities
- Design, build, and maintain scalable ETL/ELT pipelines for processing large volumes of automotive test data.
- Develop high-quality Python-based solutions using advanced object-oriented programming concepts and best practices.
- Handle complex data ingestion workflows, including processing domain-specific file formats such as .sydata, .h5, and other testing data formats.
- Collaborate with cross-functional teams to support data-driven solutions within automotive testing environments.
- Implement multiprocessing and concurrency techniques to optimize system performance and data processing efficiency.
- Contribute to the architecture and development of distributed systems, ensuring scalability and reliability.
- Deploy and manage applications in Windows Server environments, including service deployment and automation scripting.
- Integrate CI/CD pipelines using tools such as GitHub Actions to streamline development and deployment processes.
- Monitor and maintain system performance using logging frameworks and visualization tools such as Grafana.
- Work with various database technologies, ensuring efficient data storage, retrieval, and management.
- Manage network-based file systems, including NAS interactions and SMB/CIFS protocols.
- Total Experience Required:
- Senior-level experience in data engineering, preferably within automotive or testing domains, with proven expertise in enterprise software development.
- Educational Requirements:
- Bachelor’s or Master’s de