Senior Data Engineer: Design & Architecture

Apply Job Type Full-time Description Job Type Full-time Description K2United is an organization that houses two distinct, national, customer-facing brands tied together by a shared purpose: setting the standard for an extraordinary workplace. Through our brands, K2Share and CareerSafe, we provide advisory services in cyber risk management and online education for workforce readiness. Our four core values define how we show up every day: Respect Others – We lead with respect, building trust and connection. Internally Driven – We are relentlessly compelled to accomplish our objectives. Collaborative Innovation – We create by listening, sharing, and working together. Client Success – We hold our client’s mission as our own. We believe in people who are accountable, curious, and motivated to make an impact that matters. Our programs make a big impact—CareerSafe supports 2 million+ users per year, and K2Share delivers IT support on federal systems. You’ll have the opportunity to make a difference in a mission-driven, small-business environment that values professional growth and work/life balance. K2Share is seeking an experienced Senior Data Engineer to join our Federal Team. This is a senior-level role for a practitioner who sits at the intersection of federal cybersecurity compliance data, big-data engineering on AWS, and AI-first development practices. You will design, architect, and build the data foundation that powers integrated governance, risk, compliance, and scoring solutions for federal agency clients. About You You are a senior data engineer who moves comfortably between designing a relational schema for audit-grade compliance data, building the analytical pipelines that turn it into insight, and using AI assistants as a daily part of how you build. You think in terms of system design - not just code - and you bring the discipline to weigh tradeoffs, document design choices, and defend them in front of technical and non-technical reviewers alike. You have direct experience with OSCAL or other structured compliance schemas and you understand why round-trip fidelity matters for data that must stand up to audit. You think about schema decisions and what they will need to support five years out, not just the next sprint. And you treat AI tooling as an accelerator that has already changed the engineering job, not a future trend you are watching from the sidelines. Your Impact The Senior Data Engineer will own the data engineering function on K2Share's Federal Team, partnering with technical and product leadership to deliver data products that support mission-critical decision-making for federal agency clients. Design and build relational data layers that handle OSCAL and other structured compliance data - including ingestion, validation, transformation, and export workflows that preserve fidelity to source schemas across the full data lifecycle Design and maintain data models that support governance, risk, compliance, scoring, and reporting workflows for federal cybersecurity programs, with OSCAL as the connective layer across them - including long-term retention and archival policies that align with federal recordkeeping and audit requirements Design and build big-data processing pipelines on Databricks (PySpark, Delta Lake, Unity Catalog) that normalize cybersecurity data from across federal agency environments and produce analytical layers for trend analysis, executive reporting, and cross-program insights Optimize data systems for performance and cost - identifying I/O and compute bottlenecks, scaling compute responsibly, and balancing throughput against the cost discipline federal engagements require Architect, build and maintain AWS data infrastructure that meets federal security and operational requirements - working across services such as S3, Bedrock, Lambda, Fargate, and EC2 in support of compliance and analytical workloads Design and implement audit-ready data primitives - change capture, access controls, validation, and lineage — that support agency reporting and continuous monitoring needs Lead AI-first development and responsible AI deployment on the data team - using AI development tools as a standard part of the engineering loop, prototyping AI-assisted compliance workflows, and designing the production AI systems behind them (RAG architectures, vector store management, conversational agents, prompt and output guardrails, and evaluation pipelines), in alignment with federal AI governance guidance (OMB, NIST AI RMF) Engage with federal agency stakeholders and internal teams during requirement discovery, delivery, and ongoing support — translating compliance needs into data products and customer feedback into improvements Requirements 5+ years of production data engineering experience, with a track record of designing and owning data systems end-to-end Strong relational database expertise with PostgreSQL or equivalent — schema design at scale, indexing and partitioning strategy, access control, and patterns for handling semi-structured data including JSON. Strong system design and architecture instincts - able to translate business and compliance requirements into data system designs, document tradeoffs, and lead design reviews with technical and non-technical stakeholders. 3+ years of big-data processing on Databricks, Spark, or equivalent — PySpark, Delta Lake, Unity Catalog, and medallion (bronze/silver/gold) architecture patterns Strong AWS experience including S3, Bedrock, Lambda, Fargate, EC2, relational database services, change-data-capture services, serverless compute, IAM, and KMS — ideally in GovCloud or other regulated-cloud environments Strong Python development skills, including data manipulation libraries such as pandas, for ETL, transformation, and analytical workflows Proficient with Git and modern version-control practices — branching strategies, code review discipline, and collaborative workflows in a team setting Experience working with structured external schemas - OSCAL or similar standards-based data — including the discipline of preserving fidelity through transformation Demonstrated focus on data system optimization - identifying I/O bottlenecks, remediating performance issues, balancing cost and performance, and scaling compute responsibly Schema evolution discipline - migration strategy, backward compatibility, change-data-capture-friendly design, and the operational rigor of running production schemas under change control Experience with async data processing patterns — task queues, message-based pipelines, idempotent task design Active use of AI development tools as a routine part of the engineering workflow, with informed views on where they accelerate and where they need supervision Familiarity with responsible AI deployment patterns — RAG architectures, vector databases, embedding management, prompt and output guardrails, and evaluation methods Working knowledge of federal cybersecurity frameworks: FISMA, NIST RMF, NIST SP 800-53, NIST CSF Demonstrated ability to interpret regulatory and policy guidance and translate it into product or data-product requirements Comfort working across ambiguous, fast-moving federal programs with minimal supervision and strong collaborative instincts Preferred Knowledge, Skills, and Abilities Direct OSCAL experience — schema implementation, document handling, version migration, or extension namespace design Direct experience supporting U.S. Government federal civilian agency clients Familiarity with FedRAMP authorization processes, ATO lifecycle management, and 3PAO assessment data needs Familiarity with the Microsoft Power stack (Power BI, Power Query, Power Apps, Power Automate) for federal client reporting and workflow automation Experience with Delta Sharing, cross-environment Databricks integration, or modern data-sharing patterns Familiarity with MITRE ATT&CK, STIX/TAXII, or threat-intelligence data integration Experience building or operating data products for compliance, audit, or risk-management use cases Education Bachelor’s Degree in Computer Science, Data Engineering, Management Information Systems, Cybersecurity, or equivalent from an accredited institution Security Clearance Must hold, or be eligible to obtain and maintain, a Federal Security Clearance or Public Trust Benefits: We’re invested in the people who make our success possible. As a K2United employee, you’ll enjoy a comprehensive set of benefits and opportunities that meet you where you are. These benefits include: 401(k) plan with employer matching contributions Low-cost, comprehensive medical benefits for employees and their families Flexibility for those needing time off for jury duty, voting, military leave, etc. Paid time off Wellness stipend program (includes fitness reimbursement program) Tuition stipend Casual dress work environment Technical training and certifications as required Any of our CareerSafe Online training courses for free to employees and their immediate family Applicants must be willing to take a drug test and submit to a credit and background investigation as part of the selection process. The U.S. government restricts access by Foreign Nationals to certain types of technology and technical data. Consequently, this posting is intended only for U.S. citizens. K2United, LLC is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, sexual orientation, gender identity, disability, or protected Veteran status. This job description is not an exhaustive list of job responsibilities. K2United management reserves the right to change or alter this job description at any time without notice. Salary Description $120,000 - $140,000