Job Title

Company DescriptionThreatXIntel is a growing Cybersecurity, IT Staffing, and Consulting company delivering end-to-end technology and security solutions. Our services include cloud security, web and mobile application security testing, DevSecOps, vulnerability assessments, IT consulting, and professional staffing services.We support global corporate clients by hiring and deploying skilled professionals across IT and cybersecurity domains while helping organizations strengthen security, optimize operations, and scale efficiently. ThreatXIntel is committed to enabling business growth through secure, reliable, and high-quality technology solutions.Role OverviewWe are looking for a Freelance Data Engineer with strong expertise in building and optimizing scalable ETL/ELT pipelines for batch and real-time data processing.Key Responsibilities:Design, develop, and maintain ETL/ELT data pipelinesBuild batch and real-time ingestion, transformation, and loading workflowsWork with Spark (PySpark/Scala) and streaming technologies like Kafka/FlinkDevelop and optimize scalable data architectures including data lakes, data warehouses, and streaming platformsImprove performance of Spark jobs, SQL queries, and data workflows for speed, efficiency, and cost optimizationImplement data quality checks, monitoring, and alerting systemsRequired Skills:Strong proficiency in Python and SQLExperience with Apache Spark (Spark SQL, DataFrames, Streaming)Hands-on experience with Kafka or Pub/SubFamiliarity with GCP and/or Azure data servicesKnowledge of BigQuery, Snowflake, Redshift, and NoSQL databasesExperience with Airflow, Databricks, Docker, and Kubernetes is a plus