fix: relax filtering of heading elements with classnames that include the word "header"

[inhumantsar]

This removes header from unlikely and adds it to positive in an attempt to avoid filtering legitimate heading elements.

It does seem to improve parsing generally, even capturing some previously ignored metadata, but it does introduce a few unwanted artifacts.

Closes #855 and will likely have merge conflicts with #867 and #866

[gijsk]

Hi @inhumantsar ! Thanks for investigating this. Did you mean to mark this as a work in progress and/or would you like feedback on this at this point?

[inhumantsar]

Did you mean to mark this as a work in progress and/or would you like feedback on this at this point?

It should be complete but wanted to get the other PRs in before calling it ready.

I'll rebase it and make sure it's all good tonight or tomorrow, then mark it for review

[inhumantsar]

ok so i had a chance to refresh my memory. i put this into draft until the PR with all of the publishedTime test case changes were in so that this PR wouldn't also include them.

unambiguously positive impacts:

• removes a suggested stories section from cnet
• adds a headings and review score information to engadget
• removes "Supported by" from nytimes-2
• adds dt elements to google-sre-book-1 (definitions seem a bit pointless without the terms they're defining!)
• adds section headings to nytimes-4
• captures the byline for wordpress

ambiguously positive impacts:

• captures the subheading and author information for buzzfeed-1 but adds images (the URLs for which appear to be broken) and caption "Facebook" (as in the source of the image) to them.
• replaces "Contents" in the excerpt field for mercurial with the page title.
• adds the page title to mercurial (good thing) twice (not so good).
• now identifies an author for simplyfound-1, but it captures this along with the date and several newlines: Joe Wee \n \n \n Monday, February 29, 2016 @ 11:10 PM UTC
• correctly identifies the author for heraldsun-1, which was previously capturing the author from a related stories block but it captures it as by:\n\t\t\t\t\t\t\t\t\t\t Laurie Oakes
• adds <meta> elements in the body of yahoo-3

negative impacts:

• adds related coverage blocks to yahoo-3 as well as nytimes-1 and -2

another issue is that jsdom and JSDOMParser yield different results in testing for simplyfound-1:

  1) Test pages
       simplyfound-1
         jsdom
           should extract expected content:

      AssertionError: #readability-page-1 > div:nth-child(2) > p:nth-child(2) > i:nth-child(1)(in: ``<i></i> <span>
                <a href="http://fakehost/latest?author=joewee" rel="index,follow"><span>Joe Wee</span></a>&nbsp; <i></i> </span> &nbsp;&nbsp;<i></i> <span> Monday, February 29, 2016 @ 11:10 PM UTC </span> ``): expected 'i' to deeply equal '#text(The Raspberry Pi Foundation sta…'
      + expected - actual

      -i
      +#text(The Raspberry Pi Foundation started by a handful of volunteers in 2012 when they released the original Raspberry Pi 256MB Model B without knowing what to expect. In a short four-year period they have grown to over sixty full-time employees and have shipped over )


  2) Test pages
       simplyfound-1
         jsdom
           should extract expected byline:

      AssertionError: expected 'Joe Wee       \n          \n         …' to deeply equal 'Joe Wee       \n          \n         …'
      + expected - actual

      -Joe Wee       
      +Joe Wee       
                 
                   
                     Monday, February 29, 2016 @ 11:10 PM UTC
      

  3) Test pages
       simplyfound-1
         JSDOMParser
           should extract expected byline:
     AssertionError: expected null to deeply equal 'Joe Wee       \n          \n         …'

i can probably deal with these less-than-ideal captures with some simple heuristics but not sure if that should get its own PR. i don't know where to start with the jsdom vs JSDOMParser issue though.

[juliomuhlbauer]

let's get this merged!