Ask for a project walkthrough
Choose one relevant piece of work and ask the person to explain the problem, their own contribution and what happened after delivery. Go deeper on one decision instead of skimming a long list of technologies. Respect confidentiality; a sanitized architecture or explanation can still be useful.
Explore the simplest alternative
Ask what they would try before reaching for a complex model or multi-step agent. Strong answers connect technical choices to the actual constraints. Listen for a willingness to simplify, compare options and say what they do not yet know.
Discuss evaluation early
Ask how they would choose representative examples and spot meaningful failures. For an assistant, explore incomplete information and unexpected requests. For automation, ask about duplicates and partial failures. Look for a plan that fits the actual workflow.
Follow the system into production
Talk about deployment, monitoring, updates and ownership. Who notices a problem? What can be rolled back? What information is needed to diagnose an issue? The goal is to understand their thinking, not to demand a particular tool.
Give communication a real context
Ask for a short explanation to a non-technical product owner. Then add a change in priorities or an unclear requirement. Look for questions, explicit assumptions and a practical next step, rather than a polished monologue.
Use a shared scorecard
Agree on criteria before the interviews: relevant work, technical judgment, evaluation, collaboration and ownership. Record evidence beside each criterion. Keep comparisons connected to the role and avoid treating confidence or familiarity as a substitute for capability.
Put it into practice.
Turn the useful questions into a project brief you can keep, edit and share.
Create your brief