Usecase description

This usecase aims to use Microsoft's Azure Playfab telemetry tools for exporting data and generate audio description. By setting up an hook in the PlayFab dashboard, any game using this tool can easily implement GameToSpeech. It demonstrates that, following this method, audio descirption for live streamed games can be easily achieve on a lot of games. (Please note that GameToSpeech is not affiliated with Microsoft).

Video

Start of transcription.

Hi, I’m David Libeau. I’m a PhD student working on GameToSpeech, a system that will provide audio description for games.
Today, we gonna use Playfab, a Microsoft telemetry tool that is used in many games for data analysis.
So, here we are in the Unity game engine. I made a very simple game with only a cube. I also imported the Playfab SDK. It is installed.
In my scene I have a Playfab object with a script that is asking two values, a GameToSpeech access key and a channel for authentication purposes.
Now, let’s check what is happening in my script.
We can find the two variables and a bunch of stuff that let your game connect to Playfab. In the end of the script, we have an event called “OnAudioDescriptionEvent”. Our two variables are used here, but two other variables are asked when this event is triggered: a type and a value. We also add a “audio_description” tag when sending these data to Playfab.
Now, let’s get back to our scene.
We have a cube with its own cube script. Let’s open it. Here, we have a simple “OnTriggerEnter” function. So, when the player hit the cube, it calls the “OnAudioDescriptionEvent” function of the Playfab script with “collide” and our object name as parameters.
Now, let’s launch the game. But first, I will display the console.
Here, we are. In the console, we can already see that we are connected to Playfab.
Now, I am going to walk in the cube. In the console, we can see that audio description events have been sent. It’s working!
Now, let’s setup how we transmit Playfab data to the GameToSpeech API. I have just logged in the Playfab dashboard, then click on your game, then go to “Data” under “Analyze”, and “Webhooks”. With webhooks, it’s super easy to send data to somewhere else on the web. Here I already have an example hook. You will have to put the endpoint URL. Here I put “api.gametospeech.com/debug”, because I was testing. But we will put something like “/hub/playfab” in the end. I have also supplementary headers, but the most important part is the filters. Here I am filtering the event name in order to have only audio description events.
And that’s all. Data are sent to the GameToSpeech server! Here is a sample: we find our access token, our channel, our data type and value as expected. Thanks to that, spectators could enjoy audio description.

End of transcription.

Source code

Browse source code

Contact

If you need more information about this usecase, you can contact David Libeau by sending an email here: phd@davidlibeau.fr

References list