Media Sources: 15 Political Parties: 5 Politicians: 160 Articles: 76,183 Coverage: 752,393 Period: Oct 1 2019 to Jan 18 2021

About

MediaBias is a service offered as a public good, that measures political bias in the New Zealand media, using a machine learning model that evaluates the sentiment of sentences from news articles and blog posts containing the names of Members of Parliament (MPs) and political parties.

Whilst the results are preliminary, it is the the largest analysis of political bias in the New Zealand media to date.

The current analysis is performed on 76,183 articles, published by 15 New Zealand media sources, from Oct 1, 2019 to Jan 18, 2021. From these articles, a total of 574,279 sentences were identified that mentioned MPs and political parties. 464,270 (81%) of these sentences contributed towards the bias and political leaning scores. All 752,393 mentions of MPs and political parties contributed towards the measurement of how much coverage each political party receives.

For more details about how the current analysis is performed, please see how it works.

Limitations

The limitations of the current methodology include:

  • To measure bias you need a neutral point of comparison, however there isn't one when you are comparing the sentiment of parties based on time. Scandals from different parties may average each other out over time, however, they also may not. A party may also rightly deserve criticism from the media, so negative sentiment about a party does not automatically mean bias.
  • The analysis was performed on political articles published when the Labour Party was in power, the results may change if run on articles published when the National Party was in power.
  • The method of assigning sentence sentiment scores to parties has limitations:
    • Sentences in which multiple parties are mentioned are excluded.
    • The MP or party in the sentence could be criticising a non-political entity.
    • The sentence could be a part of a block quote, which the article author affirms or negates after the block quote. Block quotes of third party sources are common in political blogs.
  • Parties with less coverage are more susceptible to errors in the process of assigning sentiment, as there are less examples to average out errors in the sentiment data.
  • Opposition parties could in theory be assigned more negative sentiment because it is their role to be critical of the Government.
  • Limitations with data sources:
    • The BFD: does not include premium content.
    • Newsroom: does not include Newsroom Pro content.
    • TVNZ, NewstalkZB and Newshub: only includes content from news articles, not radio or television broadcasts, which is a significant proportion of the content published by these three organisations.
    • Scoop: currently includes all political content, including press releases. In the future it may be better to only include articles that are authored or curated by their editorial team (e.g. articles on the front page).
Roadmap
The following is a rough outline of our plan for further development:
  • Include articles from one year before the 2017 general election in the analysis, so that the dataset contains one year of articles when the National and Labour parties were in Government.
  • To aid with the model's explainability, add the ability to view the sentences, extracted MPs and sentiment scores that contribute to the results for each media source.
  • Include reporters in the analysis.
  • Train a relationship extraction model that is able to handle more complexity in assigning sentence sentiment to parties.
  • Automatically detect important events and add them as annotations to the timeline view.
  • Perform an analysis of political bias based on events discussed by media sources, which would address limitations due to a lack of neutral point of comparison.