AI Engineer — KontexIQ

Work arrangement: Remote / Hybrid Employment type: Permanent Experience: 2+ years

About KontexIQ

KontexIQ is an AI-powered software delivery intelligence platform for teams using Azure DevOps and Jira. It helps teams review requirements before development, identify ambiguity and missing acceptance criteria, generate practical test scenarios, and make risk-based testing decisions. By connecting requirements with relevant bugs, tests, and delivery history, KontexIQ provides recommendations supported by rationale, confidence, and evidence.

About the Role

We’re looking for an AI Engineer to help build the AI applications behind KontexIQ. Your primary focus will be developing requirement-analysis and testing workflows grounded in relevant project information. You’ll work across retrieval, LLM integration, evaluation, backend services, and production monitoring, collaborating with product and engineering colleagues to turn software-delivery needs into dependable capabilities. This is a hands-on role for someone who has built AI-powered applications beyond prototypes and enjoys investigating how they behave in practice.

What You’ll Do

• Build AI agents and workflows that analyze requirements, identify potential gaps, and support testing decisions.
• Develop retrieval pipelines that select relevant requirements, discussions, bugs, tests, and project history.
• Ground findings in supporting evidence and provide clear explanations that users can review.
• Handle incomplete or conflicting information by making uncertainty and unresolved questions visible.
• Develop evaluations for accuracy, relevance, consistency, and important failure modes.
• Integrate AI capabilities with Azure DevOps, Jira, and other services through APIs, and build the supporting backend services.
• Monitor production quality, latency, reliability, and cost; investigate failures across prompts, retrieval, tools, and application logic.
• Own assigned capabilities from development through deployment and ongoing improvement.

What You’ll Bring

• 2+ years of professional software or AI engineering experience, with hands-on experience building LLM-powered applications beyond experimentation.
• Strong programming skills in Python or another backend language, with experience developing APIs and backend services.
• Practical experience with LLM APIs, prompt design, structured outputs, and tool/function calling.
• Experience with retrieval systems, embeddings, vector or semantic search, and improving retrieval quality.
• Experience evaluating AI outputs and investigating incorrect or unreliable results.
• Familiarity with API authentication, third-party integrations, version control, testing, and CI/CD.
• Clear communication, strong problem-solving skills, and comfort working with evolving product requirements.

Good to Have

• Experience with multi-step agents or model and tool orchestration frameworks.
• Familiarity with Azure DevOps, Jira, GitHub, or software-delivery workflows.
• Experience with knowledge graphs or combining multiple retrieval approaches.
• Experience building evaluation datasets and using LLM tracing or observability tools.
• Familiarity with FastAPI, cloud deployment, or Kubernetes.
• Experience with software testing, requirements engineering, or enterprise AI products.

- How We Work We value engineers who investigate failures carefully, question assumptions, and use evidence to guide improvements. You’ll be a strong fit if you enjoy making AI outputs useful and understandable, incorporating feedback from users, and taking responsibility for how your work performs in production.

Why KontexIQ? You’ll work on a practical challenge: helping software teams use their existing project knowledge to improve requirements and testing decisions. The role offers the opportunity to contribute to product capabilities spanning AI workflows, retrieval, evaluation, and engineering integration. Your work will connect directly to the questions Product, Engineering, and QA teams face during delivery.

How to Apply

Send your CV and, if available, a GitHub profile, portfolio, or brief description of a relevant AI application you’ve built to: [Application link or email address] We’d like to hear what you built, your contribution, and how you evaluated or improved its reliability. Product familiarity is a mandatory part of our hiring process. Before applying, please review kontexiq.com to understand the product and the problems it addresses. If shortlisted, you’ll be required to try KontexIQ before your interview and bring one observation or question to the discussion.