Twitch Chat Poll Visualizer

What if anybody could visualize impromptu Twitch poll results?

Every #100Devs class starts with a Question of the Day, it's nothing too serious, but it gets a vast number of responses, and one can only be so accurate estimating by watching a blazing fast Twitch chat.

So my plan was to take my downloaded Twitch chat J​SON files and generate animated graphs from them. Now I'm well aware there are likely Twitch Extensions or even the official features - Twitch Poll - that can do this already, but the problem is that both of these require the streamer to trigger, so if the streamer just wanted to ask a question off the top of their mind, they wouldn't likely go through this process.

Another Twitch Userscript

..is NOT what this is! I will confess that the idea had entered my mind for a brief second, but the idea of doing all that scraping to get the chat messages while it's going by, and the fact that the speed controls would be bound to Twitch's 2x meant it would be a tedious process both to develop and to use.

So this is a plain website that accepts a JSON upload, containing all the Twitch chat messages, and uses Chart.js to visualize them.

Customizable Form

Before I even got to the visualization I needed to get this form working of course: it'd obviously accept a file via <input type="file">, but in addition to that it would need to allow the playback speed to be adjusted, the playback to be paused and unpaused accordingly, the start and end times to visualize, and finally the parameters for the different options.

I decided there would only be two options, both for simplicity and the fact that Leon's Question of the Day has yet to have more then two answers. So each question would have a color, a title, and a shell-like list of keywords that should count toward that answer. I can already hear you, this is a wildly inaccurate way to count votes, not only do I know this, but I also don't care about accuracy. This is not going to be determining anything at all, so I feel just to visualize it's fine if there are a number of false positives.

Visualization

Finally onto the core of the program: the actual visualization. Now I already knew from my experience with my #100Devs Asset Manager that there's a lot of data in these chat messages, so to be performant each tick I would remember the index of the last message I processed, and pick up from there the next. Now I was ambitious, I wanted two graphs animated at once: a pie chart AND a line graph, which thankfully weren't too difficult to create.

The Pie chart was as simple as generating two numbers, while the line graph required a bit more work, creating the data objects for each line whenever a vote was received for one. Another thing I did have to consider before generating data for either was how to handle duplicate or changed votes, which I decided to do by only counting the users latest vote. This did mean that one could change their vote, which had no impact on the side effects for the pie chart, but for the line graph what would happen is that the users previous vote would disappear entirely from the graph, as if they had only voted once.

Perhaps at a later date I could expand my vote tracking to include previous votes, which would allow me to generate the data needed to make the line chart more accurate, but at this point I both doubted anybody would notice and anybody would care. Though the fact that I'd eventually be posting a blog article about this very issue would probably bring the most attention to it comparatively.

1.8 Million Words

Finally it was working as expected, I provided a starting time, and ending time, and it would count the messages between those times as it "played" at the given rate, updating the charts as it went along.

Now as I intended to share these on Discord, I had to design the layout to be more video-friendly. Firstly was grabbing the discord background color, making the charts vertical instead of horizontal, and finally after hitting "Play" for there to be a moment of delay with nothing onscreen, allowing for the video to start with a empty frame, resulting a nearly camouflaged thumbnail.

I did discover a unique behavior while making the charts disappear for a moment, no matter what I did - opacity: 0, display: none, etc - for whatever reason the charts themselves would not go away instantly, and at some times stayed on screen for 10s of seconds! Even directly setting the <canvas> css display: none had no effect! Now my theory was that it had to be Canvas.js doing this, so I decided to invert the displays: it would initially start with no charts, then show up after a few second delay when the playback began. Of course when playback stopped the issue persisted, but that was long after the video itself would had ended, so I had no concern for it.

Inefficient Pipeline

While I was able to record the first video with no issues, I wanted these to be easily replicable - not down to the pixel, but down to the results - so I implemented quick form autocompleting from the URL GET parameters, allowing me to simply copy down the URL and save all the form settings, so if I wanted to rerecord the videos I wouldn't need to remember or retype the entire form again.

All done you might think, which is what I was thinking also, but the process of recording the actual video was now the most tedious part of this process. So my next goal was to use the MediaDevices.getDisplayMedia() function to have the browser itself record the visualization. The main issue blocking this was the fact that it records the entire window/screen, so the only way to get UI-less videos is to go fullscreen, which would lead to landscape videos when I wanted portrait videos. Thankfully I have some experience with media streams, so had already thought of how to crop it - calling Element.getBoundingClientRect() on my desired element to record and using that to crop it. I hadn't enough experience to do so off the top of my head though, so this feature would have to be left to another day.

Epilogue

At this point the visualizer met nearly all my needs, it handles a variable amount of options, properly counts when people change their vote, and is reliable enough to record and post the more interesting results to my Twitter:

As I didn't initially create the project with Twitter as a target, I had gone ahead and make a number of various other theme options - Dark, Light, Discord, Twitter Dim, GitHub Dark, etc.

I had yet to automate the recording process itself, while I had technically got the MediaDevices.getDisplayMedia() approach working, unfortunately the quality was lower then if I were to manually record it with software such as SimpleScreenRecorder. Meaning any automation of recording would either have to be system-automation, or I'd have to figure out how to increase the quality of the browser-based recorders.

There is one last piece of UI that I felt would be beneficial: a navigable timeline. Of course I would probably implement this via a <input type="range">, so nothing extravagant, but this would allow navigation between the start and end times with ease.

Future

I had envisioned a page that you can visit and see all the visualizations I've made from #100Devs - not all of them were interesting enough for Twitter, and sometimes there are multiple visualizable-moments per class, so I've only posted the most interesting one.

The user would then be able to not only play them as normal videos, but also click on them, visiting the page itself allowing them to at playback the visualization itself - and with that new timeline navigation UI - scroll through the visualization like a video.

Until I actually get around to implementing such a page, the above Tweets & Setting links will have to be suffice for now.