tl;dr
Using our state-of-the-art Kanon legal AI classifier, we evaluated the fairness of the terms of service of 112 of the world’s most well-known companies and found Nintendo to top the chart by number of consumer unfriendly clauses, closely followed by Xiaomi and Via Transportation.
Lopsided clauses are rife
Research suggests as few as 9% to 1% of consumers actually read terms of service.
Those numbers are not surprising. Terms of service are notoriously verbose and chock-full of jargon. Many terms of service could not be read — let alone understood — by your average passenger on the Clapham omnibus in under an hour. Roblox’s terms of service, for example, are almost as long as Hamlet, Shakespeare’s longest play.
Nevertheless, as inaccessible as they may be, terms of service are important to read — they are meant to govern our relationships with service providers and suppliers, from what happens when we think they’ve wronged us to what we can and cannot use their products for.
Last year, we, a team of seasoned legal and AI experts, sought out to make legal documents like terms of service a little less inaccessible by building highly accurate, lightweight and sovereign AI models for the legal domain.
As of today, we offer Kanon Universal Classifier, a state-of-the-art legal zero-shot classification model that can accurately pull-out one-sided clauses from lengthy contracts in mere milliseconds, in addition to benchmarking-breaking information extraction and retrieval models, with first-class embedding and generative models slated for release within the year.
In furtherance of our mission to make law and legal technology accessible to all, we’ve applied Kanon Universal Classifier to all the English-language terms of service in the European Union’s Digital Services Terms and Conditions Database and are now make our findings freely available and inspectable to consumers.
Exploring consumer terms with AI
Out of the 112 terms of service we’ve analyzed, we’ve found that 44% have clauses waving the right to sue in court without first attending some form of arbitration, 91% have clauses limiting the liability of a service provider in some way and 75% have clauses obligating consumers to compensate service providers for losses.
Alarmingly, we’ve also found that at least 38% of clauses are one-sided in favor of service providers, either by granting a right to a service provider that is not granted to their consumers or by placing an obligation on their consumers that is not placed on them.
Among the consumer unfriendly clauses we identified are a number that indeed, by our reading, are potentially in breach of Australia’s new unfair contract terms regime, exposing service providers to considerable financial liability.
Note: for the sake of brevity, this plot only covers the seven most common consumer unfriendly clause types. You can inspect all the clauses we looked at using the data viewer shown later in this blog post or by checking out our GitHub repository.
One type of consumer unfriendly clause of particular concern to us was class action waivers, which prohibit consumers from launching class action suits against their service providers.
Class action waivers have been found to be void by the High Court of Australia under the Australian Consumer Law by virtue of being unfair. Their enforceability is also questionable in Canada, England and Wales, Scotland, Belgium and they are void (for consumers) in France and Austria.
In Australia, under the Unfair Contract Terms regime brought into effect in late 2023, companies can be fined A$50 million or more depending on their valuation and turnover if they propose, apply, rely upon or purport to apply or rely upon an unfair contract term, which can include class action waivers.
Despite that, over the course of our survey, we have identified a number of service providers that include class action waivers in the terms of service presented to Australian users, including BlueSky, Pinterest and Spotify.
Indeed, Spotify’s class action waiver, which reads as follows, was explicitly included in their Australian terms of service:
WHERE PERMITTED UNDER THE APPLICABLE LAW, YOU AND SPOTIFY AGREE THAT EACH MAY BRING CLAIMS AGAINST THE OTHER ONLY IN YOUR OR ITS INDIVIDUAL CAPACITY AND NOT AS A PLAINTIFF OR CLASS MEMBER IN ANY PURPORTED CLASS OR REPRESENTATIVE ACTION. Unless both you and Spotify agree, no arbitrator or judge may consolidate more than one person's claims or otherwise preside over any form of a representative or class proceeding.
This clause was found by cross-referencing the arbitration clauses of European and Australian terms of service. Done by hand, it could take weeks to parse and classify the 112 consumer terms we analyzed, however, thanks to Kanon, we managed to do that in less than 12 minutes.
You can explore all of the consumer unfriendly and potentially unfair clauses we identified, including Spotify’s class action waiver, using the data viewer below:
Consumer unfriendly clause explorer
No clauses of this type present in the document.
In addition to our data viewer, we’ve also produced a radar chart allowing you to interactively compare the contractual unfairness of the terms of service providers of your choosing. To keep things manageable, we’ve presented service providers’ scores (the calculation of which is described later in this blog post) for the seven most common consumer unfriendly clauses, instead of all the clauses we’ve looked at.
Compare companies by ranks
To score service providers by the potential unfairness of their contracts, we computed, for each of the consumer unfriendly clauses we looked at, the frequency of those clauses in each service provider’s contract relative to the number of clauses in their contract.
We then converted those scores into ranks against the scores of other service providers for each clause type, enabling like-for-like comparisons between the rankings of service providers by clause type.
Using this simple percentile ranking method, we found Nintendo’s terms of service to top the charts in consumer unfriendliness, which we confirmed to be the case through manual review.
Particularly egregious terms include:
- The obligation to resolve most claims through binding arbitration (though users can technically opt out of this by notifying Nintendo in writing within 30 days of becoming a user by mailing their Redmond HQ) even where a consumer would otherwise have had the right to sue in court and have a jury trial.
- Waiver of the right to file a class action lawsuit against Nintendo.
- Acceptance of ‘all risks and responsibility’ associated with unauthorized access to consumers’ accounts, with Nintendo accepting no liability for damages whatsoever to the fullest extent permitted by law.
- The right for Nintendo to perpetually commercially exploit any content you upload to their services for any purpose, ‘including for promotional or marketing purposes’, without compensation.
- Automatic assignment of complete ownership over any intellectual property rights in any ideas, questions ‘or other information about [Nintendo’s] Services’ when submitted ‘via [their] Services or otherwise’.
- A cap on Nintendo’s liability to however much a consumer has paid them, with consumers in turn fully indemnifying Nintendo and related parties for their losses.
- The right for Nintendo to vary their terms at any time (without notice outside of simply posting them to their website and updating their last updated date), with your continued use of their services automatically and immediately constituting acceptance.
Many of Nintendo’s consumer unfriendly clauses have a good chance of being struck out as unfair, void, or otherwise unenforceable under a number of unfair contractual term regimes around the world. Notably, at least one term in their US EULA has already been the subject of intense scrutiny, namely, Nintendo’s right to render your devices unusable if you try to tamper with their services.
In addition to identifying several instances of extreme one-sidedness, we also identified, more broadly, systemic patterns in the prevalence of consumer unfriendly clauses.
Specifically, we found that the terms of social media platforms, search engines, and travel sites tended to be less consumer unfriendly than those of electronics and delivery companies. It is difficult to know with certainty the exact causes of these trends, but we suspect it may be the result of a confluence of factors, including the varying levels of risk associated with such industries and the amount of attention placed on them.
These patterns are nevertheless valuable in showing potential gaps in the fairness of online terms of service — gaps that regulators may want to focus more on.
It is worth noting, of course, that not all the agreements we analyzed were perversely lopsided.
Fairbnb, a sustainability-focused alternative to Airbinb, scored among the lowest in terms of consumer unfriendliness, and indeed, our manual review found their terms to be clear, simple, and largely fair. We especially enjoyed their inclusion of a diagram illustrating how their refund policy worked.
How we did it
The insights we uncovered through our analysis are now, thanks to the existence of Kanon, easily replicable on your own. In fact, most of our time was spent producing pretty, interactive visualizations rather than classifying clauses.
To get started with Kanon, install our API:
pip install isaacus
Next, register for an Isaacus developer account on our platform and then generate your first API key. You’ll be instantly granted US$200 in credits upon verifying your billing method.
Once you’ve got your API key, you can initialize our API client like so:
from isaacus import Isaacus
client = Isaacus("PASTE_YOUR_API_KEY_HERE")
We will now create a simple helper function to extract clauses and print them out:
def classify_and_print(
text: str,
query: str,
model: str = "kanon-universal-classifier",
) -> None:
"""Classify `text` with respect to `query` using `model` and print the results.
Args:
text (`str`): The text to classify.
query (`str`): The query to classify the text against.
model (`str`, optional): The model to use for classification. Defaults to `kanon-universal-classifier`."""
# Classify the text with respect to the query using the specified model.
classifications = client.classifications.universal.create(
model=model,
query=query,
texts=[text],
)
classification = classifications.classifications[0]
chunks = classification.chunks
score = classification.score
# Print the overall classification score.
print(
f"Overall classification score: {score * 100:.2f}%",
end="\n\n",
)
# Filter out chunks with a score below 50%.
chunks = filter(lambda c: c.score > 0.5, chunks)
# Print the chunks.
if chunks:
print("#" * 18, "Snippets with a positive classification", "#" * 18, end="\n")
for chunk in chunks:
chunk_text = chunk.text
chunk_score = chunk.score
start, end = chunk.start, chunk.end
# Print the chunk in the format:
# ---------- start char = {start} | end char = {end} | score = {chunk_score * 100}% ----------
# {chunk_text}
print(
"-" * 10,
f"start char = {start:,} | end char = {end:,} | score = {chunk_score:.2%}",
"-" * 10,
"\n",
chunk_text,
end="\n\n",
)
With our helper function ready, let’s fetch Nintendo’s terms of service as our starting point:
tos = client.get(path="https://examples.isaacus.com/nintendo-tos.txt", cast_to=str)
To extract clauses from the terms, we simply supply our helper function (which is set to use our state-of-the-art Kanon Universal Classifier zero-shot legal classifier by default) with our document and a statement about it.
For example, if we wanted to pull out arbitration clauses from the terms, you could use the statement, ‘This is an arbitration clause’, like so:
classify_and_print(tos, "This is an arbitration clause.")
Likewise, we could search for indemnities like so:
classify_and_print(tos, "This is an indemnity.")
To help our users squeeze out as much accuracy as possible from our models, we’ve already optimized our own prompts for the most common types of contractual clauses, in addition to creating ‘prompt templates’ that allow you to plug in your own descriptions of clauses while maximizing precision. We’ve made these prompts available via a purpose-built query language, the Isaacus Query Language (IQL), which you can learn more about here.
We also relied on IQL in producing our analysis. You can find the complete list of all 11 clause types and the corresponding queries used to find them below:
Looking to the future
With the help of Kanon, a surprisingly lightweight language model, we were able to compress a weeks-long manual analysis into a couple minutes with minimal loss in accuracy, as confirmed by our expert review.
We expect that process to only get easier in the months to come as we release bigger and better legal AI models to help lawyers and consumers alike turn mundane work into more enjoyable high-level analysis.
If you’re a legal technology company, law firm, or legal technology engineer interested in joining us on that journey and helping us to shape the future of legal AI, please reach out. Otherwise, you can keep track of our progress by following us on LinkedIn, X or Reddit.
All information in this blog post is of a general nature only and is not intended to be relied upon as, nor to be a substitute for, legal advice. Isaacus is a legal AI company, not a legal practice. Isaacus does not practice law or provide legal advice, and we are neither qualified nor licensed to do so. No responsibility for loss occasioned to any person acting on or refraining from action as a result of this blog post can be accepted. This blog post does not establish an attorney-client relationship.
