Roman Lutz AI Safety Engineering Lead | AI Red Teaming | Trust and Safety | Open Source

I contributed to MLCommons Jailbreak Benchmark v1.0, collaborating with Chris Knotz on implementations for leaf nodes in the jailbreak taxonomy. PyRIT provided the foundation for this work.

At the end of July, we released v1 of PyRIT, Microsoft's AI red teaming toolkit. Our new blog post and white paper, co-written with Rich Lundeen and Eugenia Kim, trace its evolution from a research prototype into a production-grade platform used to red team hundreds of generative AI products and services. PyRIT now supports repeatable, measurable, and collaborative red teaming through the Python framework, scanner CLI, CoPyRIT graphical user interface, and RAMPART. The publications also show how Volkan Kutal uses PyRIT for reproducible, auditable enterprise assessments and how MLCommons uses its attack libraries and prompt converters to power a jailbreak benchmark organized around the MLCommons Jailbreak Taxonomy. Huge kudos to my fellow PyRIT maintainers Rich Lundeen, Behnam Ousat, Hannah Westra, Justin Song, Varun Joginpalli, Toby Kohlenberg, Adrian Gavrila, Bolor-Erdene Jagdagdorj, Victor Valbuena, Spencer Schoenberg, Bashir Partovi, and Nina Chikanov.

I spoke about Red Teaming AI: Getting Started with PyRIT for Safer Generative AI Systems at PyData Seattle!

Our paper "Just Do It!? Computer-Use Agents Exhibit Blind Goal-Directedness" has been accepted at ICLR 2026! This was a collaboration between the AI Red Team and our friends at Microsoft Research AI Frontiers, based on Erfan Shayegani's summer internship. The work was also covered by 404 Media.

Tori Westerhoff and I joined Michael Kennedy on the Talk Python To Me podcast to talk about red teaming LLMs and generative AI with PyRIT.

Our new whitepaper "Taxonomy of Failure Modes in Agentic AI Systems" is out! For a high-level idea, check out this Microsoft blog.

I mentored Naijing Guo and Salma Zainana from the Stanford ICME program (together with Eric Darve and Kari Hanson) as they brought state-of-the-art jailbreaks to PyRIT. Salma implemented Tree of Attacks with Pruning and Naijing contributed the GCG (Greedy Coordinate Gradient) adversarial suffix attack — a far more involved, optimization-based jailbreak than a simple converter — expanding PyRIT's automated attack toolkit for probing generative AI systems.

I mentored Safwan Ahmed during his Microsoft internship, where he implemented two well-known jailbreaks in PyRIT. He added the single-turn Skeleton Key attack, disclosed by Mark Russinovich, and the multi-turn Crescendo orchestrator, giving red teamers reusable implementations of both.

PyRIT was a key part of Phi-3 Safety Post-Training which is highlighted in the associated paper Phi-3 Safety Post-Training: Aligning Language Models with a "Break-Fix" Cycle

One of the highlights from my perspective was going to the Microsoft //Build conference in May to talk to customers about PyRIT. After just about eight (!) years at Microsoft this was my first //Build conference. My colleagues Tori Westerhoff and Pete Bryan did an amazing job talking about the work of the AI Red Team in their session.

I have joined the AI Red Team. See these articles for some background [1], [2], and [3].

Our new paper is on ArXiv! Titled A Framework for Automated Measurement of Responsible AI Harms in Generative AI Applications, it talks about some of the ways we have been evaluating LLMs. This was a joint effort of many teams at Microsoft and Microsoft Research. I am particularly happy with the emphasis on input from domain experts. This is merely a tool to help speed up evaluations, but the actual decisions about mitigations and whether a system is deployed remains (and should remain) with humans.

The new Fairlearn paper is now in the Journal for Machine Learning Research (Open Source Software section)! It captures our change from being a project under Microsoft governance to being a true open source project with open governance. As of today, half the maintainers are employed by Microsoft (including myself). Also, the focus of the project has shifted significantly since the original whitepaper. Back then, the Python toolkit was the main focus whereas now the educational materials are being prioritized. This aims to acknowledge the sociotechnical nature of fairness.

I mentored Kevin Spiekermann and Britney Ting from MIT on Fairlearn. They documented the modern folktables benchmarks from Ding et al. as replacements for the dated UCI Adult dataset — Kevin added documentation and a fetch function for ACSIncome and Britney added documentation for the sibling ACSPublicCoverage dataset.

We released the Responsible AI dashboard. As one of the key contributors on the engineering side I am really proud of this milestone. Of course, this is only where it really starts as we can now iterate on the first version. Make sure to try it and leave some feedback! The functionality is better captured by the blog and website, but something not mentioned there that I am really excited about is that we pulled this off in the open on GitHub. That means anyone can see what goes into this, ask for features, or even contribute bugfixes. Doing impactful work is awesome, but seeing the recognition in the entire company takes this to a whole different level. For example, I have seen tweets about this by Microsoft CTO Kevin Scott and Chief Scientific Officer Eric Horvitz.

I mentored Lilian Kourti and Xiaoye Yuan from the Stanford ICME program (together with Matt Lungren, Kari Hanson, and Trevor Hastie) on fairness in chest X-ray classification, studying how deep-learning diagnostic models perform across patient subgroups.

I mentored Davide Giovanardi, Lauren Pendo, and Andra Fehmiu from the Stanford ICME program (together with Kari Hanson and Mary Wootters) on Fairlearn. Together they implemented bounded group loss via exponentiated gradient, extending Fairlearn's fairness reductions to regression problems.

I mentored Parul Gupta and Abdul from UMass Amherst (together with Miro Dudík) on Fairlearn. Abdul added the Equal Opportunity constraint while Parul contributed additional reduction constraints and grid-search improvements to the fairness algorithms.

I gave a keynote on fairness in algorithms for The Good AI, covering how to assess and mitigate unfairness in machine learning systems with Fairlearn.

For a little while now I have been working on Responsible AI at Microsoft. Now that Sarah Bird announced our tools at Ignite I can finally point to our tools publicly. A lot of my time over the past months went into Fairlearn, our open source toolkit for fairness assessment and unfairness mitigation. We just released v0.3.0, so there is a lot more to come in the next months. I will be in Vancouver for NeurIPS in December to demo our tools around fairness and interpretability. Talk to me if you will be there!

I mentored Emilio Alfonso Venancio from the Instituto Tecnológico Autónomo de México on Azure Machine Learning.

Participating at MIT`s Breaking the Mold Hackathon for Inclusion was truly a blessing. With so many truly difficult problems to tackle, it is fantastic to see all the ideas people came up with. Big shoutout to MIT for organizing this, Microsoft for the venue (and encouraging me to go!), and Amazon for sending two inspiring mentors for my team all the way from Seattle! Thanks also to my team for creating a creative environment where everybody could express their ideas. I learned a ton from all of you, and winning 3rd prize tops it all off. I hope everybody takes some time to think about Machine Learning Bias. With ML becoming increasingly prevalent, it is more important than ever to take bias into account.

After a year on the Office team, I moved to the Azure Machine Learning team. We are building the infrastructure and services from scratch using lots of open source (e.g., Kubernetes, Linux, .NET core).

I participated in the Hacking Bias in ML workshop at Microsoft`s New England Research and Development Center (which happens to be my office, too). My group specifically looked at gender bias in text through word embeddings. We found lots of evidence of gender bias, e.g., some words are generally more used in connection with men ("smart"), some more with women ("lovely"). You can play around with the tool resulting from the workshop here.

I have joined the Docs team within Microsoft Office. We enhance collaboration capabilities through the Share feature in all Office apps.

I took part in Baden-Württemberg Exchange between University of Ulm and University of Massachusetts Amherst.

After a year of studying in the Mathematics Bachelor`s program with a minor in Computer Science I decided to swap major and minor. I still graduated with a BSc in Computer Science with honors. The courses covered basics in Systems, AI, and Theory.

The goal of my Bachelor`s thesis was to implement the Adaptive Large Neighborhood Search (ALNS) heuristic and possibly come up with improvements. ALNS was described first by S. Ropke and D. Pisinger and is based on P. Shaw`s Large Neighborhood Search. The idea is that some problems are difficult to solve with basic local search algorithms because of a tightly constrained search space. Small changes to a solution will rarely bring improvements. As a consequence, LNS and ALNS change larger parts based on different heuristics. For this thesis, I received the innoWake Award 2015. innoWake was a software modernization company based in Austin, TX and had a number of branch offices including one in Germany. They have since been acquired by Deloitte.