Graph Communications Bot Media SDK
The Real-time Media Platform enables bots to interact with Microsoft Teams calls and meetings using real-time voice, video and screen sharing. This is an advanced capability which allows the bot to send and receive voice and video content frame by frame. The bot has "raw" access to the voice, video and screen sharing media streams. (Bots which process media themselves are called application-hosted media bots, as opposed to simpler service-hosted media bots that rely on the Real-time Media platform for all media processing.)
For example, in a 1:1 call with a bot, as the user speaks, the bot will receive 50 audio frames per second, with each frame containing 20 milliseconds (ms) of audio. An application-hosted media bot can perform real-time speech recognition as the audio frames are received, rather than having to wait for a recording after the user has stopped speaking. The bot can also send and receive high-definition-resolution video, including video-based screen sharing content.
The platform provides a simple "socket"-like API for the bot to send and receive media, and handles the real-time encoding and decoding of audio/video packets, using codecs such as SILK and G.722 for audio and H.264 for video. The platform also handles all media packet encryption/decryption and packet network transmission automatically, so the bot just needs to concern itself with the actual audio/video content. A real-time media bot may participate in 1:1 calls as well as meetings with multiple participants.
Microsoft Real-Time Media Platform for Bots API ("Media Access API")
Accompanying Documentation
The Media Access API is offered under the terms of the Microsoft APIs Terms of Use (https://docs.microsoft.com/en-us/legal/microsoft-apis/terms-of-use) and this accompanying documentation.
"Media Access API" means (i) any form of machine accessible application programming interface that Microsoft makes publicly available which provides access to a Microsoft Offering that includes media content (e.g., audio or video), including all associated tools, elements, components and executables therein, (ii) any Microsoft sample code that enables interactions with a Microsoft Offering, and (iii) documentation that Microsoft makes available to help enable your access to the Media Access API.
Additional Restriction: You may NOT use the Media Access API to record or otherwise persist media content from calls or meetings that your Application accesses, or data derived from that media content.
Select a class on the left to dive into the reference docs