Link Unfurling

How I generated these neat embeds!

This feature was quite the journey for me, I started off thinking I'd just add some meta tags, and be done with it. I was wrong.

As I have no video or app content, I will focus on static textual and image content only, and I have not tested these methods on every site that performs Link Unfurling.****

What is Link Unfurling?

Link Unfurling is the process that sites perform on links to generate content relative to the link in question. Most popular websites implement some form of link unfurling, and while there are standards - <meta> tags, the Open Graph, oEmbed format, and even site-specific standards, there's no substitute for going to a site and manually testing your link to see how the site treats it.

Thankfully there are various preview generator sites, with varying degrees of accuracy:

...and many more, though these are the ones I used, along with taking my URL and posting/previewing it on the sites manually.

Another fortunate thing is that while how they are parsed can differ from site to site, the actual code needed to get this working is extremely simple.

If you're curious to what I'm actually using, there's nothing stopping you from inspecting the HTML of this very page!

OpenGraph

I used these OpenGraph tags:

<meta property="og:type" content="website"/>
<meta property="og:title" content={meta.title} />
<meta property="og:description" content={meta.description}/>
<meta property="og:url" content={website}/>
<meta property="og:image" content={`${website}/embeds/embed-image.gif`}/>
<meta property="og:image:secure_url" content={`${website}/embeds/embed-image.gif`}/>

If this syntax looks strange, it's because I'm using React here, but it has zero impact on the end result.

I told you the code was simple, so most of these are straightforward - this is a website, title, description, and url are plain strings, then of course the images.

Now these URLs needs to be absolute, not relative, but aside from that they should be reasonably sized, though it seems each platform prefers their own aspect radio and resolution, I found a 2:1 aspect ratio - resolution of 800x400 - to work for most of them.

The astute amount you would've noticed that my images are GIFs, and yes, you can use GIFs...depending on the platform. Noticing a pattern yet? Yeah at least Twitter does not support animating GIFs, they take the first frame and display it as a static image.

oEmbed

When I found this I had thought I found the one true standard - a single standard that everything works with. Of course that's not what I found, and I honestly can't tell if platforms use this when the other tags exist, but just to be safe I did use it.

{
  "type": "link",
  "version": "1.0",
  "author_name": "Rascal Two",
  "author_url": "https://rascaltwo.com",
  "provider_name": "Your next Software Engineer!",
  "provider_url": "https://rascaltwo.com",
  "thumbnail_url": "https://rascaltwo.com/embeds/embed-image.gif"
}

These are again, mostly straightforward to read, though as you can tell I have not followed the specification exactly in relation to my provider and author usage, I will explain this decision once I get to the platform differences.

Twitter Card

Twitter has actually become a commonly used standard by other platforms, so even if you have no intention of being posted to Twitter, these can still provide additional data:

<meta name="twitter:card" content="summary_large_image" />
<meta name="twitter:domain" content={website.split('//')[1]} />
<meta name="twitter:title" content={name}/>
<meta name="twitter:description" content={meta.description} />
<meta name="twitter:image" content={`${website}/embeds/embed-twitter.jpg`}/>
<meta name="twitter:site" content="@RealRascalTwo" />
<meta name="twitter:creator" content="@RealRascalTwo" />
<meta name="twitter:label1" content="<a href="https://github.com/">GitHub</a>" />
<meta name="twitter:data1" content="https://github.com/RascalTwo" />

Following the set pattern, most of this is not only easily readable, but also repeated. The domain does need to be only the domain, and in my case I linked a JPG for the image instead of the GIF, along with additional labels.

Twitter allow you to define fields, suffixed with a digit, so the field I've defined has a heading of GitHub and content of my GitHub URL.

GIF perils

If you're as ambitious as I am and really want your GIFs to work, I will advise you to keep you to focus heavily on minimizing your GIF file size, as the larger you go the more platforms will simply refuse to load your image - even if to display a single frame.

I was able to get my desired animation at a 3.1MB while having it still work on platforms - of course delayed for platforms such as Discord that have serve the image raw.

The Differences

Each platform has it's varying degrees of documentation describing it's Link Unfurling process, but I've yet to find a complete and accurate breakdown, so will describe the ones that I had to go out of my way to accommodate.

Twitter

I had initially been using PNGs for my static images, figuring why not, they were of reasonable size and all the other platforms worked with them flawlessly...except Twitter.

For whatever reason the PNGs I had attempted to use not only would fail, but would sometimes succeed, leading to me pushing my code to the server only to discover that in fact, it was a false positive.

Through sheer frustration I had decided to generate some JPGs with near zero quality to just see if it would work...and it did. I incrementally increase the quality, first 25%, then 75%, and finally up to 100% and it was still working. Of course I assumed another false positive, so to the server my updated assets went...and it was still working.

Now it's entirely possible the PNGs I was generating were bad, too large, or a plethora of different reasons that were actually my fault, but the experience of false positives, no messages in the Twitter Card Validator about why my image had failed, it simply left me frustrated, and even as I had finally gotten something stable for Twitter, still leaves me frustrated not knowing what I was doing wrong.

Discord

I still do not know what rules Discord uses for it's Link Unfurling, but I do know that if you want full embeds with hotlinked Provider, Author, and a custom embed color, you need to use only this combination of HTML & oEmbed data:

<meta property="og:title" content={meta.title} />
<meta property="og:description" content={meta.description} />
<meta property="og:image" content={`${website}/embeds/embed-image.gif`} />
<meta name="twitter:card" content="summary_large_image"></meta>
<link type="application/json+oembed" href={`${website}/embeds/oembed-${short}.json`} />

But how did you only use these HTML tags for Discord specifically you may ask? Well as I'm using Next.js, I have access to a configurable redirects method that can return rules for where to redirect users based on various things, including User-Agent headers:

return [{
  source: '/',
  destination: '/embed/discord',
  has: [{ type: 'header', key: 'User-Agent', value: '(.*Discordbot.*)' }],
  permanent: false
}, {
  source: '/blog',
  destination: '/blog/embed/discord',
  has: [{ type: 'header', key: 'User-Agent', value: '(.*Discordbot.*)' }],
  permanent: false
}, {
  source: '/blog/embed/:slug',
  destination: '/blog/embed/discord/:slug',
  has: [{ type: 'header', key: 'User-Agent', value: '(.*Discordbot.*)' }],
  permanent: false
}]

I know this is a bit verbose, but it here's what the rules mean: if any request has a User-Agent header with a value matching the regex value provided - containing Discordbot - it'll be redirected to a special discord variant of the page in question with the HTML needed.

Of course if you're not using Next.js this isn't an option for you, a vanilla approach would be creating and sharing this discord subroute when you post to discord, and having way to redirect users from this Discord-landing-page to the content they're actually looking for.

Testing

One thing that will become immediately obvious if you attempt to use any of the listed validators it that they all accept URLs, which is fine for once you get your changes pushed, but when you're doing rapid development, this really slows down the process.

Thankfully there are various ways to expose your local machine to the internet so these sites can access your local site, the method I used was starting up ngrok, a command-line utility that exposes a local port to a randomly generated - or one of your choosing - subdomain of ngrok.io, allowing you to provide this URL to the service and it to work.

Dynamic

Making these images dynamically is something that's actually quite a popular thing to do, so I won't get into as much detail seeing there are dozens of other resources that can explain the process better.

My process was similar to most of them though, I created a page specifically for taking screenshots of, and wrote a playwright script to visit all of these pages, saving the screenshots - additionally generating the oEmbed JSONs accordingly.

You can see this posts embed page here

Future

At this point you might have noticed that only my homepage embed image is a GIF, while the rest are all static, unfortunately I feel the benefit of my animation is not enough for me to invest the necessary time to automate the generation of GIFs...though that hasn't stopped me before, so only time will tell!

Rascal Two's Blograscaltwo.com

Link Unfurling

Rascal Two's Blog
rascaltwo.com