Project Description:Develop scalable data collection, storage, and distribution platform to house data from vendors, research providers, exchanges, PBs, and web-scraping. Make data available to systematic & fundamental PMs, and enterprise functions: Ops, Risk, Trading, and Compliance. Develop internal data products and analyticsResponsibilities:Web scraping using scripts/APIs/ToolsHelp build and maintain greenfield data platform running on Snowflake and AWSUnderstand the existing pipelines and enhance pipelines for the new requirements.Onboarding new data providersData migration projectsMandatory Skills:• SQL - senior level• Python - senior level• Streamlit Expertise• Linux• Containerization(Docker, Kubernetes)• Good communication skills• AWS• Dev ops skills (K8s, Docker, Jenkins)Nice-to-Have Skills:• Market Data Projects/ Capital markets exp• Snowflake is a big plus• AirflowLanguages:English: B2 Upper Intermediate
Job Title
Data Engineer for Market Data Projects (with Streamlit Expertise)