Wikipedia, as a graph

How it is made

• Fetch. Every revision of the article, with its wikitext, time, editor and edit summary; the Wikidata item; the articles that link in. Anonymous editors are recorded as anonymous, never by address.
• Parse. A parser (admin/build/wikitext.py, no model) lifts the references out, turns the markup into text, and splits it into sentences. One sentence is one statement; its id is a hash of its words, so the same sentence keeps its id across revisions and an edit is a set difference.
• Class. By wording only: a record (a number, a date, an identifier), a plain stated fact, an attributed statement (a named party said, reported or estimated it), a quote, or hedged wording. The class says how the article speaks, not whether it is right.
• Join. A reference is matched to this site's sources by its address and to a party by its host. A statement there and a claim here that cite the same captured page meet at that page.
• Keep. The data is data/wikipedia/; the graph of each article is in the ontology's wikipedia scope (data/ontology.json), with its own kinds and verbs, joined to the story graph by captured_as.

Wikidata and DBpedia

Wikidata is the structured sister of Wikipedia: one item per thing, with typed claims, edited in the open with the same history as an article. DBpedia is the older extraction of the same facts from Wikipedia's infoboxes, built from periodic dumps. This site reads both for the articles it watches, keeps the item's own history (on each article's evolution page), and looks the parties of its own graph up on Wikidata so that a party here can carry the item that is already there, rather than a new identifier: 95 parties looked up, 25 items confirmed by hand, 2 proposed and waiting. The lookups are in wikidata-links.json; the build shows only confirmed items.

Learned from them so far: an item is created by whichever language edition gets there first (this one from the Finnish Wikipedia, two hours after the English article); its claims are typed and sourced one by one, which is the shape this site's own attribution rows take; and an identifier that exists already is the one to use.

For Wikipedia's editors

This site does not edit Wikipedia and does not ask to be cited by it. What it offers the article's editors, and the rules it holds itself to, are on one page; its reading of the article as a reviewer would read it, finding by finding against the captures, is on each article's review page.

What this is for

Three things. A reader who wants the facts of a story, each with its evidence and without the markup, the edit wars and the list at the bottom. A newsroom that can see what Wikipedia says, what it rests on, and what changed since it last looked, in one view. And, in time, a measure: a source's trust shown by how often its statements are corrected, against other publications', rather than assumed.

The text of each article is Wikipedia's, CC BY-SA 4.0, by its contributors, with the revision id as the attribution; the pages and data of this section that carry it are under the same licence.