How ShopWeight measures app speed
What a check does
A check loads an ecommerce store's homepage five times in Google's Lighthouse, with the settings of a mid-range phone on a mobile connection. Lighthouse records how much processing time and download size each script on the page uses. Nothing is installed on the store and nothing on it changes.
How scripts are matched to apps
We match each script to an app by its web address. We use Shopify's own app file paths, a list of known app domains we keep by hand, and an open list of web technology fingerprints. Scripts we can't match are never counted toward any app.
Why we show bands
The same page takes a different amount of work on every load. We take the middle result of the five loads and place it in one of four bands: light (under 100 ms), moderate (100 to 300 ms), heavy (300 ms to 1 second) or very heavy (over 1 second). An exact number would suggest more precision than a page load allows.
Checks run in the cloud
Checks run on Google Cloud machines. Lighthouse measures each machine's speed before every load. We scale the results to one reference machine so checks from different machines compare fairly.
What we don't measure
Processing time shows how much work an app makes a shopper's phone do. It doesn't show what the app earns for the store. A heavy app can still be worth keeping.
How the index is built
The App Weight Index uses the most recent check of each store. An app appears only after it has been measured on at least 20 stores, because fewer stores give numbers too uneven to publish about someone's product. The typical figure is the middle store's result. Stores are never named in the index.
Real-shopper speed data
Reports also show figures from Google's Chrome UX Report when Google publishes them for the store. They come from real Chrome visits on phones over the past 28 days. They cover the whole page. Only our lab check can single out each app.
How apps are compared across stores
A report compares each app with that app's most recent result on the other stores in the index. It shows the comparison only for apps measured on at least 20 stores.
Corrections
If you make an app or run a store and think a result is wrong, email shopweight@2visions.com. We'll recheck it and correct anything we got wrong.