GenAI in production · apply · about 10 minutes

The internal assistant that can do things

A team demos an internal assistant. It indexes every document on the company wiki and shared drive, answers questions with citations, and can call three tools: send an email as the user, create a Jira ticket, and look up an employee record. The system prompt contains the service account token for the Jira API. Quality was judged by the team trying it for a week and liking it. A prompt-injection filter runs on user input. It is proposed for company-wide rollout on Monday. You have been asked for a view.

Give the view. Rank what you would fix, and be clear about which items block Monday and which do not.

Write your answer somewhere else — on paper, in a mail draft, out loud to someone. Then come back and mark it against the rubric. The rubric is disputable: if you think a line is wrong for your situation, you are probably right, and noticing that is worth more than the tick.

Mark it against the rubric

Blocks on the token in the system prompt — it should be treated as already leaked The system prompt is not a secret
Blocks on indexing everything: retrieval must be filtered by the asker's identity, before retrieval The vector store inherits every access problem
Names send-email-as-the-user plus ingestion of untrusted documents as the injection path that matters Prompt injection is the unsolved one
Says the input filter is a mitigation and cannot be the only control on sending mail A guardrail is a mitigation, not a boundary
Calls "we tried it for a week and liked it" not a measurement, and asks for a fixed eval set "It looks good" is not a measurement
Scopes the tools rather than rejecting the project — and separates Monday blockers from the backlog Excessive agency is a permissions question

Read against OWASP Top 10 for Large Language Model Applications, 2025, NIST AI 600-1 — Generative AI Profile (AI RMF), Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (arXiv:2005.11401), Hu et al., LoRA: Low-Rank Adaptation of Large Language Models (arXiv:2106.09685) on 2026-10-16. Regulations and frameworks change; the date is there so you can tell whether this has kept up.

The OWASP Top 10 for LLM Applications is © the OWASP Foundation and published under a Creative Commons licence; NIST AI 600-1 is a US Government work in the public domain. These cards are independent commentary written in our own words. ytyk.org is not affiliated with or endorsed by the OWASP Foundation or NIST.