Prototype
AI that works in desktop software through MCP
An MCP server through which an AI assistant sees the screen of a program running in a virtual machine and works its mouse and keyboard – where the program has no API or the API can't do everything.
Built for a Windows accounting program, but it works with any desktop program.
- Project
- Own prototype
- Our role
- The whole solution
- Years
- 2026
- Technologies
- MCP, Windows virtual machine, screen capture, mouse and keyboard control
The task
Let an AI assistant do a task in a desktop program when the program's API doesn't cover it.
The solution
An MCP server through which an AI assistant sees the screen of a program running in a virtual machine and uses the mouse and keyboard.
- Nine tools, with scaling of screen coordinates.
- Built to operate a Windows accounting program's desktop client for work its API doesn't cover.
- It works with any desktop program.
Done by AI: AI assistant
MCP server
Nine tools
The desktop program in a virtual machine
The AI sees the screen and uses the mouse and keyboard
Results
- A working prototype with nine tools for an AI assistant.
- It works with any desktop program running in a virtual machine.
Related services and solutions
Want a similar solution?
A few sentences are enough – we'll reply and talk through how we could help.