Applied AI engineer for local service businesses in Warsaw

Applied AI engineer for local service businesses in Warsaw

Warsaw Stacjonarnie
O

Описание:

We’re looking for a hands-on Applied AI Engineer to own the quality of Handle’s AI agent. You’ll expand and run the agent’s evals and benchmarks, drive the hill-climb loop on its skills, and help make the agent reliable, useful, and competitive. Задачи:

Build and own the eval suite: scenarios, rubrics, graders, benchmarks, and quality gates. Form research hypotheses, run controlled experiments, and separate model effects from prompt, tool, skill, and agent-structure effects. Version skills and changes, rerun the full evaluation suite, diagnose regressions, and follow improvements through to verified results. Evaluate conversation quality, style, user understanding, and task reliability. Test meaningful new models and assess portability, failure modes, and when a simpler single-model strategy is better. Recommend agent or skill restructuring when repeated failures show that local fixes are not enough. Требования:

Proven experience in building and running evals for an LLM product in production, ability to write the code. Deep interest in agent behavior, model capabilities, and evaluation. Enough technical fluency to run experiments, inspect failures, and collaborate closely with engineers. Ability to move between theory and concrete product problems. Proactive taste: you notice patterns, propose better approaches, and follow them through. Будет плюсом: Hands-on post‑training or fine‑tuning of open-weight models Условия:

At On The Spot, we’re creating a workplace people would want to stay in: supportive, well‑organized, and fair in terms of conditions and benefits. We’ll share more about our benefit package during the interviews.

Applied AI engineer for local service businesses in Warsaw – pracodawca: On The Spot Development

Orca Security w Warszawie to doskonały pracodawca, który oferuje wyjątkowe możliwości rozwoju zawodowego w dynamicznym środowisku technologicznym. Pracując w zespole Sensor, będziesz miał szansę na bezpośredni wpływ na kierunek techniczny projektu, a także korzystać z kultury współpracy i innowacji, która sprzyja kreatywności i zaangażowaniu. Dodatkowo, firma zapewnia elastyczne godziny pracy oraz wsparcie w nauce i rozwoju umiejętności, co czyni ją idealnym miejscem dla ambitnych inżynierów oprogramowania.

O

Dane kontaktowe:

Zespół rekrutacyjny On The Spot Development