mirrors
/
devdocs
mirror of https://github.com/freeCodeCamp/devdocs
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Refactoring and cleaning up

Browse Source
pull/986/head
Jasper van Merle 6 years ago
parent cd43632a2c
commit 41e0a138a6
142 changed files with 530 additions and 676 deletions
Show Stats Download Patch File Download Diff File

32
docs/Scraper-Reference.md
Unescape View File

@ -187,39 +187,41 @@ More information about how filters work is available on the [Filter Reference](.
## Keeping scrapers up-to-date
In order to keep scrapers up-to-date the `get_latest_version(options, &block)` method should be overridden. If `self.release` is defined, this should return the latest version of the documentation. If `self.release` is not defined, it should return the Epoch time when the documentation was last modified. If the documentation will never change, simply return `1.0.0`. The result of this method is periodically reported in a "Documentation versions report" issue which helps maintainers keep track of outdated documentations.
In order to keep scrapers up-to-date the `get_latest_version(opts)` method should be overridden. If `self.release` is defined, this should return the latest version of the documentation. If `self.release` is not defined, it should return the Epoch time when the documentation was last modified. If the documentation will never change, simply return `1.0.0`. The result of this method is periodically reported in a "Documentation versions report" issue which helps maintainers keep track of outdated documentations.
To make life easier, there are a few utility methods that you can use in `get_latest_version`:
* `fetch(url, options, &block)`
* `fetch(url, opts)`
Makes a GET request to the url and calls `&block` with the body.
Makes a GET request to the url and returns the response body.
Example: [lib/docs/scrapers/bash.rb](../lib/docs/scrapers/bash.rb)
* `fetch_doc(url, options, &block)`
* `fetch_doc(url, opts)`
Makes a GET request to the url and calls `&block` with the HTML body converted to a Nokogiri document.
Makes a GET request to the url and returns the HTML body converted to a Nokogiri document.
Example: [lib/docs/scrapers/git.rb](../lib/docs/scrapers/git.rb)
* `fetch_json(url, options, &block)`
* `fetch_json(url, opts)`
Makes a GET request to the url and calls `&block` with the JSON body converted to a dictionary.
* `get_npm_version(package, options, &block)`
Makes a GET request to the url and returns the JSON body converted to a dictionary.
Calls `&block` with the latest version of the given npm package.
Example: [lib/docs/scrapers/mdn/mdn.rb](../lib/docs/scrapers/mdn/mdn.rb)
* `get_npm_version(package, opts)`
Returns the latest version of the given npm package.
Example: [lib/docs/scrapers/bower.rb](../lib/docs/scrapers/bower.rb)
* `get_latest_github_release(owner, repo, options, &block)`
* `get_latest_github_release(owner, repo, opts)`
Calls `&block` with the latest GitHub release of the given repository ([format](https://developer.github.com/v3/repos/releases/#get-the-latest-release)).
Returns the latest GitHub release of the given repository ([format](https://developer.github.com/v3/repos/releases/#get-the-latest-release)).
Example: [lib/docs/scrapers/jsdoc.rb](../lib/docs/scrapers/jsdoc.rb)
* `get_github_tags(owner, repo, options, &block)`
* `get_github_tags(owner, repo, opts)`
Calls `&block` with the list of tags on the given repository ([format](https://developer.github.com/v3/repos/#list-tags)).
Returns the list of tags on the given repository ([format](https://developer.github.com/v3/repos/#list-tags)).
Example: [lib/docs/scrapers/liquid.rb](../lib/docs/scrapers/liquid.rb)
* `get_github_file_contents(owner, repo, path, options, &block)`
* `get_github_file_contents(owner, repo, path, opts)`
Calls `&block` with the contents of the requested file in the default branch of the given repository.
Returns the contents of the requested file in the default branch of the given repository.
Example: [lib/docs/scrapers/minitest.rb](../lib/docs/scrapers/minitest.rb)

71
lib/docs/core/doc.rb
Unescape View File

@ -164,16 +164,15 @@ module Docs
raise NotImplementedError
end
def get_scraper_version(opts, &block)
def get_scraper_version(opts)
if self.class.method_defined?(:options) and !options[:release].nil?
block.call options[:release]
options[:release]
else
# If options[:release] does not exist, we return the Epoch timestamp of when the doc was last modified in DevDocs production
fetch_json('https://devdocs.io/docs.json', opts) do |json|
items = json.select {|item| item['name'] == self.class.name}
items = items.map {|item| item['mtime']}
block.call items.max
end
json = fetch_json('https://devdocs.io/docs.json', opts)
items = json.select {|item| item['name'] == self.class.name}
items = items.map {|item| item['mtime']}
items.max
end
end
@ -181,7 +180,7 @@ module Docs
# If options[:release] is defined, it should be in the same format
# If options[:release] is not defined, it should return the Epoch timestamp of when the documentation was last updated
# If the docs will never change, simply return '1.0.0'
def get_latest_version(options, &block)
def get_latest_version(opts)
raise NotImplementedError
end
@ -216,55 +215,49 @@ module Docs
# Utility methods for get_latest_version
#
def fetch(url, options, &block)
def fetch(url, opts)
headers = {}
if options.key?(:github_token) and url.start_with?('https://api.github.com/')
headers['Authorization'] = "token #{options[:github_token]}"
if opts.key?(:github_token) and url.start_with?('https://api.github.com/')
headers['Authorization'] = "token #{opts[:github_token]}"
end
options[:logger].debug("Fetching #{url}")
opts[:logger].debug("Fetching #{url}")
response = Request.run(url, { headers: headers })
Request.run(url, { headers: headers }) do |response|
if response.success?
block.call response.body
else
options[:logger].error("Couldn't fetch #{url} (response code #{response.code})")
block.call nil
end
if response.success?
response.body
else
opts[:logger].error("Couldn't fetch #{url} (response code #{response.code})")
nil
end
end
def fetch_doc(url, options, &block)
fetch(url, options) do |body|
block.call Nokogiri::HTML.parse(body, nil, 'UTF-8')
end
def fetch_doc(url, opts)
body = fetch(url, opts)
Nokogiri::HTML.parse(body, nil, 'UTF-8')
end
def fetch_json(url, options, &block)
fetch(url, options) do |body|
block.call JSON.parse(body)
end
def fetch_json(url, opts)
JSON.parse fetch(url, opts)
end
def get_npm_version(package, options, &block)
fetch_json("https://registry.npmjs.com/#{package}", options) do |json|
block.call json['dist-tags']['latest']
end
def get_npm_version(package, opts)
json = fetch_json("https://registry.npmjs.com/#{package}", opts)
json['dist-tags']['latest']
end
def get_latest_github_release(owner, repo, options, &block)
fetch_json("https://api.github.com/repos/#{owner}/#{repo}/releases/latest", options, &block)
def get_latest_github_release(owner, repo, opts)
fetch_json("https://api.github.com/repos/#{owner}/#{repo}/releases/latest", opts)
end
def get_github_tags(owner, repo, options, &block)
fetch_json("https://api.github.com/repos/#{owner}/#{repo}/tags", options, &block)
def get_github_tags(owner, repo, opts)
fetch_json("https://api.github.com/repos/#{owner}/#{repo}/tags", opts)
end
def get_github_file_contents(owner, repo, path, options, &block)
fetch_json("https://api.github.com/repos/#{owner}/#{repo}/contents/#{path}", options) do |json|
block.call(Base64.decode64(json['content']))
end
def get_github_file_contents(owner, repo, path, opts)
json = fetch_json("https://api.github.com/repos/#{owner}/#{repo}/contents/#{path}", opts)
Base64.decode64(json['content'])
end
end
end

4
lib/docs/scrapers/angular.rb
Unescape View File

@ -155,8 +155,8 @@ module Docs
end
end
def get_latest_version(options, &block)
get_npm_version('@angular/core', options, &block)
def get_latest_version(opts)
get_npm_version('@angular/core', opts)
end
private

4
lib/docs/scrapers/angularjs.rb
Unescape View File

@ -70,8 +70,8 @@ module Docs
self.base_url = "https://code.angularjs.org/#{release}/docs/partials/"
end
def get_latest_version(options, &block)
get_npm_version('angular', options, &block)
def get_latest_version(opts)
get_npm_version('angular', opts)
end
end
end

7
lib/docs/scrapers/ansible.rb
Unescape View File

@ -88,10 +88,9 @@ module Docs
list_of_all_modules.html)
end
def get_latest_version(options, &block)
fetch_doc('https://docs.ansible.com/ansible/latest/index.html', options) do |doc|
block.call doc.at_css('.DocSiteProduct-CurrentVersion').content.strip
end
def get_latest_version(opts)
doc = fetch_doc('https://docs.ansible.com/ansible/latest/index.html', opts)
doc.at_css('.DocSiteProduct-CurrentVersion').content.strip
end
end
end

7
lib/docs/scrapers/apache.rb
Unescape View File

@ -34,10 +34,9 @@ module Docs
Licensed under the Apache License, Version 2.0.
HTML
def get_latest_version(options, &block)
fetch_doc('http://httpd.apache.org/docs/', options) do |doc|
block.call doc.at_css('#apcontents > ul a')['href'][0...-1]
end
def get_latest_version(opts)
doc = fetch_doc('http://httpd.apache.org/docs/', opts)
doc.at_css('#apcontents > ul a')['href'][0...-1]
end
end
end

9
lib/docs/scrapers/apache_pig.rb
Unescape View File

@ -43,11 +43,10 @@ module Docs
self.base_url = "https://pig.apache.org/docs/r#{release}/"
end
def get_latest_version(options, &block)
fetch_doc('https://pig.apache.org/', options) do |doc|
item = doc.at_css('div[id="menu_1.2"] > .menuitem:last-child')
block.call item.content.strip.sub(/Release /, '')
end
def get_latest_version(opts)
doc = fetch_doc('https://pig.apache.org/', opts)
item = doc.at_css('div[id="menu_1.2"] > .menuitem:last-child')
item.content.strip.sub(/Release /, '')
end
end
end

8
lib/docs/scrapers/async.rb
Unescape View File

@ -18,11 +18,9 @@ module Docs
Licensed under the MIT License.
HTML
def get_latest_version(options, &block)
fetch_doc('https://caolan.github.io/async/', options) do |doc|
version = doc.at_css('#version-dropdown > a').content.strip[1..-1]
block.call version
end
def get_latest_version(opts)
doc = fetch_doc('https://caolan.github.io/async/', opts)
doc.at_css('#version-dropdown > a').content.strip[1..-1]
end
end
end

7
lib/docs/scrapers/babel.rb
Unescape View File

@ -23,10 +23,9 @@ module Docs
'<div></div>'
end
def get_latest_version(options, &block)
fetch_doc('https://babeljs.io/docs/en/', options) do |doc|
block.call doc.at_css('a[href="/versions"] > h3').content
end
def get_latest_version(opts)
doc = fetch_doc('https://babeljs.io/docs/en/', opts)
doc.at_css('a[href="/versions"] > h3').content
end
end
end

8
lib/docs/scrapers/backbone.rb
Unescape View File

@ -21,11 +21,9 @@ module Docs
Licensed under the MIT License.
HTML
def get_latest_version(options, &block)
fetch_doc('https://backbonejs.org/', options) do |doc|
version = doc.at_css('.version').content
block.call version[1...-1]
end
def get_latest_version(opts)
doc = fetch_doc('https://backbonejs.org/', opts)
doc.at_css('.version').content[1...-1]
end
end
end

8
lib/docs/scrapers/bash.rb
Unescape View File

@ -18,11 +18,9 @@ module Docs
Licensed under the GNU Free Documentation License.
HTML
def get_latest_version(options, &block)
fetch('https://www.gnu.org/software/bash/manual/html_node/index.html', options) do |body|
version = body.scan(/, Version ([0-9.]+)/)[0][0]
block.call version[0...-1]
end
def get_latest_version(opts)
body = fetch('https://www.gnu.org/software/bash/manual/html_node/index.html', opts)
body.scan(/, Version ([0-9.]+)/)[0][0][0...-1]
end
end
end

4
lib/docs/scrapers/bluebird.rb
Unescape View File

@ -19,8 +19,8 @@ module Docs
Licensed under the MIT License.
HTML
def get_latest_version(options, &block)
get_npm_version('bluebird', options, &block)
def get_latest_version(opts)
get_npm_version('bluebird', opts)
end
end
end

7
lib/docs/scrapers/bootstrap.rb
Unescape View File

@ -35,10 +35,9 @@ module Docs
options[:only] = %w(getting-started/ css/ components/ javascript/)
end
def get_latest_version(options, &block)
fetch_doc('https://getbootstrap.com/', options) do |doc|
block.call doc.at_css('#bd-versions').content.strip[1..-1]
end
def get_latest_version(opts)
doc = fetch_doc('https://getbootstrap.com/', opts)
doc.at_css('#bd-versions').content.strip[1..-1]
end
end
end

9
lib/docs/scrapers/bottle.rb
Unescape View File

@ -28,11 +28,10 @@ module Docs
self.base_url = "https://bottlepy.org/docs/#{self.version}/"
end
def get_latest_version(options, &block)
fetch_doc('https://bottlepy.org/docs/stable/', options) do |doc|
label = doc.at_css('.sphinxsidebarwrapper > ul > li > b')
block.call label.content.sub(/Bottle /, '')
end
def get_latest_version(opts)
doc = fetch_doc('https://bottlepy.org/docs/stable/', opts)
label = doc.at_css('.sphinxsidebarwrapper > ul > li > b')
label.content.sub(/Bottle /, '')
end
end
end

4
lib/docs/scrapers/bower.rb
Unescape View File

@ -20,8 +20,8 @@ module Docs
Licensed under the MIT License.
HTML
def get_latest_version(options, &block)
get_npm_version('bower', options, &block)
def get_latest_version(opts)
get_npm_version('bower', opts)
end
end
end

11
lib/docs/scrapers/c.rb
Unescape View File

@ -26,12 +26,11 @@ module Docs
Licensed under the Creative Commons Attribution-ShareAlike Unported License v3.0.
HTML
def get_latest_version(options, &block)
fetch_doc('https://en.cppreference.com/w/Cppreference:Archives', options) do |doc|
link = doc.at_css('a[title^="File:"]')
date = link.content.scan(/(\d+)\./)[0][0]
block.call DateTime.strptime(date, '%Y%m%d').to_time.to_i
end
def get_latest_version(opts)
doc = fetch_doc('https://en.cppreference.com/w/Cppreference:Archives', opts)
link = doc.at_css('a[title^="File:"]')
date = link.content.scan(/(\d+)\./)[0][0]
DateTime.strptime(date, '%Y%m%d').to_time.to_i
end
private

7
lib/docs/scrapers/cakephp.rb
Unescape View File

@ -71,10 +71,9 @@ module Docs
self.base_url = 'https://api.cakephp.org/2.7/'
end
def get_latest_version(options, &block)
fetch_doc('https://api.cakephp.org/3.7/', options) do |doc|
block.call doc.at_css('.version-picker .dropdown-toggle').content.strip
end
def get_latest_version(opts)
doc = fetch_doc('https://api.cakephp.org/3.7/', opts)
doc.at_css('.version-picker .dropdown-toggle').content.strip
end
private

4
lib/docs/scrapers/chai.rb
Unescape View File

@ -24,8 +24,8 @@ module Docs
Licensed under the MIT License.
HTML
def get_latest_version(options, &block)
get_npm_version('chai', options, &block)
def get_latest_version(opts)
get_npm_version('chai', opts)
end
end
end

7
lib/docs/scrapers/chef.rb
Unescape View File

@ -48,10 +48,9 @@ module Docs
options[:only_patterns] = [/\A#{client_path}\//, /\A#{server_path}\//]
end
def get_latest_version(options, &block)
fetch_doc('https://downloads.chef.io/chef', options) do |doc|
block.call doc.at_css('h1.product-heading > span').content.strip
end
def get_latest_version(opts)
doc = fetch_doc('https://downloads.chef.io/chef', opts)
doc.at_css('h1.product-heading > span').content.strip
end
end
end

7
lib/docs/scrapers/clojure.rb
Unescape View File

@ -28,10 +28,9 @@ module Docs
self.base_url = 'https://clojure.github.io/clojure/branch-clojure-1.7.0/'
end
def get_latest_version(options, &block)
fetch_doc('http://clojure.github.io/clojure/index.html', options) do |doc|
block.call doc.at_css('#header-version').content[1..-1]
end
def get_latest_version(opts)
doc = fetch_doc('http://clojure.github.io/clojure/index.html', opts)
doc.at_css('#header-version').content[1..-1]
end
end
end

9
lib/docs/scrapers/cmake.rb
Unescape View File

@ -60,11 +60,10 @@ module Docs
self.base_url = 'https://cmake.org/cmake/help/v3.5/'
end
def get_latest_version(options, &block)
fetch_doc('https://cmake.org/documentation/', options) do |doc|
link = doc.at_css('.entry-content ul > li > strong > a > big')
block.call link.content.scan(/([0-9.]+)/)[0][0]
end
def get_latest_version(opts)
doc = fetch_doc('https://cmake.org/documentation/', opts)
link = doc.at_css('.entry-content ul > li > strong > a > big')
link.content.scan(/([0-9.]+)/)[0][0]
end
end
end

7
lib/docs/scrapers/codeception.rb
Unescape View File

@ -19,10 +19,9 @@ module Docs
Licensed under the MIT License.
HTML
def get_latest_version(options, &block)
fetch_doc('https://codeception.com/changelog', options) do |doc|
block.call doc.at_css('#page > h4').content
end
def get_latest_version(opts)
doc = fetch_doc('https://codeception.com/changelog', opts)
doc.at_css('#page > h4').content
end
end
end

4
lib/docs/scrapers/codeceptjs.rb
Unescape View File

@ -22,8 +22,8 @@ module Docs
Licensed under the MIT License.
HTML
def get_latest_version(options, &block)
get_npm_version('codeceptjs', options, &block)
def get_latest_version(opts)
get_npm_version('codeceptjs', opts)
end
end
end

9
lib/docs/scrapers/codeigniter.rb
Unescape View File

@ -39,11 +39,10 @@ module Docs
self.release = '3.1.8'
end
def get_latest_version(options, &block)
fetch_doc('https://codeigniter.com/user_guide/changelog.html', options) do |doc|
header = doc.at_css('#change-log h2')
block.call header.content.scan(/([0-9.]+)/)[0][0]
end
def get_latest_version(opts)
doc = fetch_doc('https://codeigniter.com/user_guide/changelog.html', opts)
header = doc.at_css('#change-log h2')
header.content.scan(/([0-9.]+)/)[0][0]
end
end
end

4
lib/docs/scrapers/coffeescript.rb
Unescape View File

@ -31,8 +31,8 @@ module Docs
options[:container] = '.container'
end
def get_latest_version(options, &block)
get_npm_version('coffeescript', options, &block)
def get_latest_version(opts)
get_npm_version('coffeescript', opts)
end
end
end

15
lib/docs/scrapers/cordova.rb
Unescape View File

@ -43,13 +43,14 @@ module Docs
self.base_url = 'https://cordova.apache.org/docs/en/6.x/'
end
def get_latest_version(options, &block)
fetch_doc('https://cordova.apache.org/docs/en/latest/', options) do |doc|
label = doc.at_css('#versionDropdown').content.strip
version = label.scan(/([0-9.]+)/)[0][0]
version = version[0...-1] if version.end_with?('.')
block.call version
end
def get_latest_version(opts)
doc = fetch_doc('https://cordova.apache.org/docs/en/latest/', opts)
label = doc.at_css('#versionDropdown').content.strip
version = label.scan(/([0-9.]+)/)[0][0]
version = version[0...-1] if version.end_with?('.')
version
end
end
end

11
lib/docs/scrapers/cpp.rb
Unescape View File

@ -35,12 +35,11 @@ module Docs
HTML
# Same as get_latest_version in lib/docs/scrapers/c.rb
def get_latest_version(options, &block)
fetch_doc('https://en.cppreference.com/w/Cppreference:Archives', options) do |doc|
link = doc.at_css('a[title^="File:"]')
date = link.content.scan(/(\d+)\./)[0][0]
block.call DateTime.strptime(date, '%Y%m%d').to_time.to_i
end
def get_latest_version(opts)
doc = fetch_doc('https://en.cppreference.com/w/Cppreference:Archives', opts)
link = doc.at_css('a[title^="File:"]')
date = link.content.scan(/(\d+)\./)[0][0]
DateTime.strptime(date, '%Y%m%d').to_time.to_i
end
private

7
lib/docs/scrapers/crystal.rb
Unescape View File

@ -35,10 +35,9 @@ module Docs
end
}
def get_latest_version(options, &block)
fetch('https://crystal-lang.org/api', options) do |body|
block.call body.scan(/Crystal Docs ([0-9.]+)/)[0][0]
end
def get_latest_version(opts)
body = fetch('https://crystal-lang.org/api', opts)
body.scan(/Crystal Docs ([0-9.]+)/)[0][0]
end
end
end

7
lib/docs/scrapers/d.rb
Unescape View File

@ -27,10 +27,9 @@ module Docs
%w(https://dlang.org/phobos/index.html https://dlang.org/spec/intro.html)
end
def get_latest_version(options, &block)
fetch_doc('https://dlang.org/changelog/', options) do |doc|
block.call doc.at_css('#content > ul > li:nth-child(2) > a')['id']
end
def get_latest_version(opts)
doc = fetch_doc('https://dlang.org/changelog/', opts)
doc.at_css('#content > ul > li:nth-child(2) > a')['id']
end
end
end

4
lib/docs/scrapers/d3.rb
Unescape View File

@ -59,8 +59,8 @@ module Docs
options[:only_patterns] = [/\.md\z/]
end
def get_latest_version(options, &block)
get_npm_version('d3', options, &block)
def get_latest_version(opts)
get_npm_version('d3', opts)
end
end
end

9
lib/docs/scrapers/dart.rb
Unescape View File

@ -32,11 +32,10 @@ module Docs
self.base_url = "https://api.dartlang.org/stable/#{release}/"
end
def get_latest_version(options, &block)
fetch_doc('https://api.dartlang.org/', options) do |doc|
label = doc.at_css('footer > span').content.strip
block.call label.sub(/Dart /, '')
end
def get_latest_version(opts)
doc = fetch_doc('https://api.dartlang.org/', opts)
label = doc.at_css('footer > span').content.strip
label.sub(/Dart /, '')
end
end
end

7
lib/docs/scrapers/django.rb
Unescape View File

@ -64,10 +64,9 @@ module Docs
self.base_url = 'https://docs.djangoproject.com/en/1.8/'
end
def get_latest_version(options, &block)
fetch_doc('https://docs.djangoproject.com/', options) do |doc|
block.call doc.at_css('#doc-versions > li.current > span > strong').content
end
def get_latest_version(opts)
doc = fetch_doc('https://docs.djangoproject.com/', opts)
doc.at_css('#doc-versions > li.current > span > strong').content
end
end
end

9
lib/docs/scrapers/docker.rb
Unescape View File

@ -138,11 +138,10 @@ module Docs
options[:only_patterns] << /\Aswarm\//
end
def get_latest_version(options, &block)
fetch_doc('https://docs.docker.com/', options) do |doc|
label = doc.at_css('.nav-container button.dropdown-toggle').content.strip
block.call label.scan(/([0-9.]+)/)[0][0]
end
def get_latest_version(opts)
doc = fetch_doc('https://docs.docker.com/', opts)
label = doc.at_css('.nav-container button.dropdown-toggle').content.strip
label.scan(/([0-9.]+)/)[0][0]
end
end
end

7
lib/docs/scrapers/dojo.rb
Unescape View File

@ -36,10 +36,9 @@ module Docs
urls.map { |url| "<a href='#{url}'>#{url}</a>" }.join
end
def get_latest_version(options, &block)
fetch_doc('https://dojotoolkit.org/api/', options) do |doc|
block.call doc.at_css('#versionSelector > option[selected]').content
end
def get_latest_version(opts)
doc = fetch_doc('https://dojotoolkit.org/api/', opts)
doc.at_css('#versionSelector > option[selected]').content
end
private

15
lib/docs/scrapers/drupal.rb
Unescape View File

@ -99,13 +99,14 @@ module Docs
]
end
def get_latest_version(options, &block)
fetch_doc('http://cgit.drupalcode.org/drupal', options) do |doc|
version = doc.at_css('td.form > form > select > option[selected]').content
version = version.scan(/([0-9.]+)/)[0][0]
version = version[0...-1] if version.end_with?('.')
block.call version
end
def get_latest_version(opts)
doc = fetch_doc('http://cgit.drupalcode.org/drupal', opts)
version = doc.at_css('td.form > form > select > option[selected]').content
version = version.scan(/([0-9.]+)/)[0][0]
version = version[0...-1] if version.end_with?('.')
version
end
end
end

7
lib/docs/scrapers/electron.rb
Unescape View File

@ -23,10 +23,9 @@ module Docs
Licensed under the MIT license.
HTML
def get_latest_version(options, &block)
fetch_doc('https://electronjs.org/docs', options) do |doc|
block.call doc.at_css('.docs-version').content
end
def get_latest_version(opts)
doc = fetch_doc('https://electronjs.org/docs', opts)
doc.at_css('.docs-version').content
end
end
end

7
lib/docs/scrapers/elixir.rb
Unescape View File

@ -98,10 +98,9 @@ module Docs
]
end
def get_latest_version(options, &block)
fetch_doc('https://hexdocs.pm/elixir/api-reference.html', options) do |doc|
block.call doc.at_css('h2.sidebar-projectVersion').content.strip[1..-1]
end
def get_latest_version(opts)
doc = fetch_doc('https://hexdocs.pm/elixir/api-reference.html', opts)
doc.at_css('h2.sidebar-projectVersion').content.strip[1..-1]
end
end
end

7
lib/docs/scrapers/ember.rb
Unescape View File

@ -57,10 +57,9 @@ module Docs
)
end
def get_latest_version(options, &block)
fetch_doc('https://emberjs.com/api/ember/release', options) do |doc|
block.call doc.at_css('.sidebar > .select-container .ember-power-select-selected-item').content.strip
end
def get_latest_version(opts)
doc = fetch_doc('https://emberjs.com/api/ember/release', opts)
doc.at_css('.sidebar > .select-container .ember-power-select-selected-item').content.strip
end
end
end

7
lib/docs/scrapers/erlang.rb
Unescape View File

@ -56,10 +56,9 @@ module Docs
self.release = '18.3'
end
def get_latest_version(options, &block)
fetch_doc('https://www.erlang.org/downloads', options) do |doc|
block.call doc.at_css('.col-lg-3 > ul > li').content.strip.sub(/OTP /, '')
end
def get_latest_version(opts)
doc = fetch_doc('https://www.erlang.org/downloads', opts)
doc.at_css('.col-lg-3 > ul > li').content.strip.sub(/OTP /, '')
end
end
end

4
lib/docs/scrapers/eslint.rb
Unescape View File

@ -21,8 +21,8 @@ module Docs
Licensed under the MIT License.
HTML
def get_latest_version(options, &block)
get_npm_version('eslint', options, &block)
def get_latest_version(opts)
get_npm_version('eslint', opts)
end
end
end

4
lib/docs/scrapers/express.rb
Unescape View File

@ -29,8 +29,8 @@ module Docs
Licensed under the Creative Commons Attribution-ShareAlike License v3.0.
HTML
def get_latest_version(options, &block)
get_npm_version('express', options, &block)
def get_latest_version(opts)
get_npm_version('express', opts)
end
end
end

7
lib/docs/scrapers/falcon.rb
Unescape View File

@ -34,10 +34,9 @@ module Docs
self.base_url = "https://falcon.readthedocs.io/en/#{self.release}/"
end
def get_latest_version(options, &block)
fetch_doc('https://falcon.readthedocs.io/en/stable/changes/index.html', options) do |doc|
block.call doc.at_css('#changelogs ul > li > a').content
end
def get_latest_version(opts)
doc = fetch_doc('https://falcon.readthedocs.io/en/stable/changes/index.html', opts)
doc.at_css('#changelogs ul > li > a').content
end
end
end

7
lib/docs/scrapers/fish.rb
Unescape View File

@ -47,10 +47,9 @@ module Docs
self.base_url = "https://fishshell.com/docs/#{version}/"
end
def get_latest_version(options, &block)
fetch_doc('http://fishshell.com/docs/current/index.html', options) do |doc|
block.call doc.at_css('#toc-index').content.scan(/([0-9.]+)/)[0][0]
end
def get_latest_version(opts)
doc = fetch_doc('http://fishshell.com/docs/current/index.html', opts)
doc.at_css('#toc-index').content.scan(/([0-9.]+)/)[0][0]
end
end
end

4
lib/docs/scrapers/flow.rb
Unescape View File

@ -19,8 +19,8 @@ module Docs
Licensed under the MIT License.
HTML
def get_latest_version(options, &block)
get_npm_version('flow-bin', options, &block)
def get_latest_version(opts)
get_npm_version('flow-bin', opts)
end
end
end

7
lib/docs/scrapers/git.rb
Unescape View File

@ -20,10 +20,9 @@ module Docs
Licensed under the GNU General Public License version 2.
HTML
def get_latest_version(options, &block)
fetch_doc('https://git-scm.com/', options) do |doc|
block.call doc.at_css('.version').content.strip
end
def get_latest_version(opts)
doc = fetch_doc('https://git-scm.com/', opts)
doc.at_css('.version').content.strip
end
end
end

9
lib/docs/scrapers/gnu/gcc.rb
Unescape View File

@ -100,11 +100,10 @@ module Docs
options[:replace_paths] = CPP_PATHS
end
def get_latest_version(options, &block)
fetch_doc('https://gcc.gnu.org/onlinedocs/', options) do |doc|
label = doc.at_css('ul > li > ul > li > a').content.strip
block.call label.scan(/([0-9.]+)/)[0][0]
end
def get_latest_version(opts)
doc = fetch_doc('https://gcc.gnu.org/onlinedocs/', opts)
label = doc.at_css('ul > li > ul > li > a').content.strip
label.scan(/([0-9.]+)/)[0][0]
end
end
end

9
lib/docs/scrapers/gnu/gnu_fortran.rb
Unescape View File

@ -26,11 +26,10 @@ module Docs
self.base_url = "https://gcc.gnu.org/onlinedocs/gcc-#{release}/gfortran/"
end
def get_latest_version(options, &block)
fetch_doc('https://gcc.gnu.org/onlinedocs/', options) do |doc|
label = doc.at_css('ul > li > ul > li > a').content.strip
block.call label.scan(/([0-9.]+)/)[0][0]
end
def get_latest_version(opts)
doc = fetch_doc('https://gcc.gnu.org/onlinedocs/', opts)
label = doc.at_css('ul > li > ul > li > a').content.strip
label.scan(/([0-9.]+)/)[0][0]
end
end
end

15
lib/docs/scrapers/go.rb
Unescape View File

@ -24,13 +24,14 @@ module Docs
Licensed under the Creative Commons Attribution License 3.0.
HTML
def get_latest_version(options, &block)
fetch_doc('https://golang.org/pkg/', options) do |doc|
footer = doc.at_css('#footer').content
version = footer.scan(/go([0-9.]+)/)[0][0]
version = version[0...-1] if version.end_with?('.')
block.call version
end
def get_latest_version(opts)
doc = fetch_doc('https://golang.org/pkg/', opts)
footer = doc.at_css('#footer').content
version = footer.scan(/go([0-9.]+)/)[0][0]
version = version[0...-1] if version.end_with?('.')
version
end
private

7
lib/docs/scrapers/godot.rb
Unescape View File

@ -38,10 +38,9 @@ module Docs
self.base_url = "http://docs.godotengine.org/en/#{self.version}/"
end
def get_latest_version(options, &block)
fetch_doc('https://docs.godotengine.org/', options) do |doc|
block.call doc.at_css('.version').content.strip
end
def get_latest_version(opts)
doc = fetch_doc('https://docs.godotengine.org/', opts)
doc.at_css('.version').content.strip
end
end
end

7
lib/docs/scrapers/graphite.rb
Unescape View File

@ -18,10 +18,9 @@ module Docs
Licensed under the Apache License, Version 2.0.
HTML
def get_latest_version(options, &block)
fetch_doc('https://graphite.readthedocs.io/en/latest/releases.html', options) do |doc|
block.call doc.at_css('#release-notes li > a').content
end
def get_latest_version(opts)
doc = fetch_doc('https://graphite.readthedocs.io/en/latest/releases.html', opts)
doc.at_css('#release-notes li > a').content
end
end
end

4
lib/docs/scrapers/grunt.rb
Unescape View File

@ -27,8 +27,8 @@ module Docs
Licensed under the MIT License.
HTML
def get_latest_version(options, &block)
get_npm_version('grunt-cli', options, &block)
def get_latest_version(opts)
get_npm_version('grunt-cli', opts)
end
end
end

4
lib/docs/scrapers/handlebars.rb
Unescape View File

@ -20,8 +20,8 @@ module Docs
Licensed under the MIT License.
HTML
def get_latest_version(options, &block)
get_npm_version('handlebars', options, &block)
def get_latest_version(opts)
get_npm_version('handlebars', opts)
end
end
end

11
lib/docs/scrapers/haskell.rb
Unescape View File

@ -69,12 +69,11 @@ module Docs
options[:only_patterns] = [/\Alibraries\//]
end
def get_latest_version(options, &block)
fetch_doc('https://downloads.haskell.org/~ghc/latest/docs/html/', options) do |doc|
links = doc.css('a').to_a
versions = links.map {|link| link['href'].scan(/ghc-([0-9.]+)/)}
block.call versions.find {|version| !version.empty?}[0][0]
end
def get_latest_version(opts)
doc = fetch_doc('https://downloads.haskell.org/~ghc/latest/docs/html/', opts)
links = doc.css('a').to_a
versions = links.map {|link| link['href'].scan(/ghc-([0-9.]+)/)}
versions.find {|version| !version.empty?}[0][0]
end
end
end

9
lib/docs/scrapers/haxe.rb
Unescape View File

@ -67,11 +67,10 @@ module Docs
self.base_url = 'https://api.haxe.org/python/'
end
def get_latest_version(options, &block)
fetch_doc('https://api.haxe.org/', options) do |doc|
label = doc.at_css('.container.main-content h1 > small').content
block.call label.sub(/version /, '')
end
def get_latest_version(opts)
doc = fetch_doc('https://api.haxe.org/', opts)
label = doc.at_css('.container.main-content h1 > small').content
label.sub(/version /, '')
end
end
end

6
lib/docs/scrapers/homebrew.rb
Unescape View File

@ -20,10 +20,8 @@ module Docs
Licensed under the BSD 2-Clause License.
HTML
def get_latest_version(options, &block)
get_latest_github_release('Homebrew', 'brew', options) do |release|
block.call release['name']
end
def get_latest_version(opts)
get_latest_github_release('Homebrew', 'brew', opts)['name']
end
end
end

4
lib/docs/scrapers/immutable.rb
Unescape View File

@ -55,8 +55,8 @@ module Docs
capybara.html
end
def get_latest_version(options, &block)
get_npm_version('immutable', options, &block)
def get_latest_version(opts)
get_npm_version('immutable', opts)
end
end
end

9
lib/docs/scrapers/influxdata.rb
Unescape View File

@ -47,11 +47,10 @@ module Docs
Licensed under the MIT license.
HTML
def get_latest_version(options, &block)
fetch_doc('https://docs.influxdata.com/influxdb/', options) do |doc|
label = doc.at_css('.navbar--current-product').content.strip
block.call label.scan(/([0-9.]+)/)[0][0]
end
def get_latest_version(opts)
doc = fetch_doc('https://docs.influxdata.com/influxdb/', opts)
label = doc.at_css('.navbar--current-product').content.strip
label.scan(/([0-9.]+)/)[0][0]
end
end
end

6
lib/docs/scrapers/jasmine.rb
Unescape View File

@ -18,10 +18,8 @@ module Docs
Licensed under the MIT License.
HTML
def get_latest_version(options, &block)
get_latest_github_release('jasmine', 'jasmine', options) do |release|
block.call release['name']
end
def get_latest_version(opts)
get_latest_github_release('jasmine', 'jasmine', opts)['name']
end
end
end

7
lib/docs/scrapers/jekyll.rb
Unescape View File

@ -29,10 +29,9 @@ module Docs
Licensed under the MIT license.
HTML
def get_latest_version(options, &block)
fetch_doc('https://jekyllrb.com/docs/', options) do |doc|
block.call doc.at_css('.meta a').content[1..-1]
end
def get_latest_version(opts)
doc = fetch_doc('https://jekyllrb.com/docs/', opts)
doc.at_css('.meta a').content[1..-1]
end
end
end

7
lib/docs/scrapers/jest.rb
Unescape View File

@ -18,10 +18,9 @@ module Docs
Licensed under the BSD License.
HTML
def get_latest_version(options, &block)
fetch_doc('https://jestjs.io/docs/en/getting-started', options) do |doc|
block.call doc.at_css('header > a > h3').content
end
def get_latest_version(opts)
doc = fetch_doc('https://jestjs.io/docs/en/getting-started', opts)
doc.at_css('header > a > h3').content
end
end
end

4
lib/docs/scrapers/jquery/jquery_core.rb
Unescape View File

@ -23,8 +23,8 @@ module Docs
/index/i
]
def get_latest_version(options, &block)
get_npm_version('jquery', options, &block)
def get_latest_version(opts)
get_npm_version('jquery', opts)
end
end
end

8
lib/docs/scrapers/jquery/jquery_mobile.rb
Unescape View File

@ -17,11 +17,9 @@ module Docs
url.sub! 'http://api.jquerymobile.com/', 'https://api.jquerymobile.com/'
end
def get_latest_version(options, &block)
fetch_doc('https://jquerymobile.com/', options) do |doc|
label = doc.at_css('.download-box > .download-option:last-child > span').content
block.call label.sub(/Version /, '')
end
def get_latest_version(opts)
doc = fetch_doc('https://jquerymobile.com/', opts)
doc.at_css('.download-box > .download-option:last-child > span').content.sub(/Version /, '')
end
end
end

4
lib/docs/scrapers/jquery/jquery_ui.rb
Unescape View File

@ -16,8 +16,8 @@ module Docs
url.sub! 'http://api.jqueryui.com/', 'https://api.jqueryui.com/'
end
def get_latest_version(options, &block)
get_npm_version('jquery-ui', options, &block)
def get_latest_version(opts)
get_npm_version('jquery-ui', opts)
end
end
end

6
lib/docs/scrapers/jsdoc.rb
Unescape View File

@ -22,10 +22,8 @@ module Docs
Licensed under the Creative Commons Attribution-ShareAlike Unported License v3.0.
HTML
def get_latest_version(options, &block)
get_latest_github_release('jsdoc3', 'jsdoc', options) do |release|
block.call release['tag_name']
end
def get_latest_version(opts)
get_latest_github_release('jsdoc3', 'jsdoc', opts)['tag_name']
end
end
end

6
lib/docs/scrapers/julia.rb
Unescape View File

@ -50,10 +50,8 @@ module Docs
html_filters.push 'julia/entries_sphinx', 'julia/clean_html_sphinx', 'sphinx/clean_html'
end
def get_latest_version(options, &block)
get_latest_github_release('JuliaLang', 'julia', options) do |release|
block.call release['tag_name'][1..-1]
end
def get_latest_version(opts)
get_latest_github_release('JuliaLang', 'julia', opts)['tag_name'][1..-1]
end
end
end

6
lib/docs/scrapers/knockout.rb
Unescape View File

@ -34,10 +34,8 @@ module Docs
Licensed under the MIT License.
HTML
def get_latest_version(options, &block)
get_latest_github_release('knockout', 'knockout', options) do |release|
block.call release['tag_name'][1..-1]
end
def get_latest_version(opts)
get_latest_github_release('knockout', 'knockout', opts)['tag_name'][1..-1]
end
end
end

4
lib/docs/scrapers/koa.rb
Unescape View File

@ -35,8 +35,8 @@ module Docs
Licensed under the MIT License.
HTML
def get_latest_version(options, &block)
get_npm_version('koa', options, &block)
def get_latest_version(opts)
get_npm_version('koa', opts)
end
end
end

6
lib/docs/scrapers/kotlin.rb
Unescape View File

@ -29,10 +29,8 @@ module Docs
Licensed under the Apache License, Version 2.0.
HTML
def get_latest_version(options, &block)
get_latest_github_release('JetBrains', 'kotlin', options) do |release|
block.call release['tag_name'][1..-1]
end
def get_latest_version(opts)
get_latest_github_release('JetBrains', 'kotlin', opts)['tag_name'][1..-1]
end
end
end

6
lib/docs/scrapers/laravel.rb
Unescape View File

@ -134,10 +134,8 @@ module Docs
end
end
def get_latest_version(options, &block)
get_latest_github_release('laravel', 'laravel', options) do |release|
block.call release['tag_name'][1..-1]
end
def get_latest_version(opts)
get_latest_github_release('laravel', 'laravel', opts)['tag_name'][1..-1]
end
end
end

9
lib/docs/scrapers/leaflet.rb
Unescape View File

@ -39,11 +39,10 @@ module Docs
self.base_url = "https://leafletjs.com/reference-#{release}.html"
end
def get_latest_version(options, &block)
fetch_doc('https://leafletjs.com/index.html', options) do |doc|
link = doc.css('ul > li > a').to_a.select {|node| node.content == 'Docs'}.first
block.call link['href'].scan(/reference-([0-9.]+)\.html/)[0][0]
end
def get_latest_version(opts)
doc = fetch_doc('https://leafletjs.com/index.html', opts)
link = doc.css('ul > li > a').to_a.select {|node| node.content == 'Docs'}.first
link['href'].scan(/reference-([0-9.]+)\.html/)[0][0]
end
end
end

9
lib/docs/scrapers/less.rb
Unescape View File

@ -22,11 +22,10 @@ module Docs
Licensed under the Creative Commons Attribution License 3.0.
HTML
def get_latest_version(options, &block)
fetch_doc('http://lesscss.org/features/', options) do |doc|
label = doc.at_css('.footer-links > li').content
block.call label.scan(/([0-9.]+)/)[0][0]
end
def get_latest_version(opts)
doc = fetch_doc('http://lesscss.org/features/', opts)
label = doc.at_css('.footer-links > li').content
label.scan(/([0-9.]+)/)[0][0]
end
end
end

7
lib/docs/scrapers/liquid.rb
Unescape View File

@ -20,10 +20,9 @@ module Docs
Licensed under the MIT License.
HTML
def get_latest_version(options, &block)
get_github_tags('Shopify', 'liquid', options) do |tags|
block.call tags[0]['name'][1..-1]
end
def get_latest_version(opts)
tags = get_github_tags('Shopify', 'liquid', opts)
tags[0]['name'][1..-1]
end
end
end

7
lib/docs/scrapers/lodash.rb
Unescape View File

@ -33,10 +33,9 @@ module Docs
self.base_url = "https://lodash.com/docs/#{release}"
end
def get_latest_version(options, &block)
fetch_doc('https://lodash.com/docs/', options) do |doc|
block.call doc.at_css('#version > option[selected]').content
end
def get_latest_version(opts)
doc = fetch_doc('https://lodash.com/docs/', opts)
doc.at_css('#version > option[selected]').content
end
end
end

7
lib/docs/scrapers/love.rb
Unescape View File

@ -40,10 +40,9 @@ module Docs
Licensed under the GNU Free Documentation License, Version 1.3.
HTML
def get_latest_version(options, &block)
fetch_doc('https://love2d.org/wiki/Version_History', options) do |doc|
block.call doc.at_css('#mw-content-text table a').content
end
def get_latest_version(opts)
doc = fetch_doc('https://love2d.org/wiki/Version_History', opts)
doc.at_css('#mw-content-text table a').content
end
end
end

7
lib/docs/scrapers/lua.rb
Unescape View File

@ -27,10 +27,9 @@ module Docs
self.base_url = 'https://www.lua.org/manual/5.1/'
end
def get_latest_version(options, &block)
fetch_doc('https://www.lua.org/manual/', options) do |doc|
block.call doc.at_css('p.menubar > a').content
end
def get_latest_version(opts)
doc = fetch_doc('https://www.lua.org/manual/', opts)
doc.at_css('p.menubar > a').content
end
end
end

4
lib/docs/scrapers/marionette.rb
Unescape View File

@ -39,8 +39,8 @@ module Docs
html_filters.push 'marionette/entries_v2'
end
def get_latest_version(options, &block)
get_npm_version('backbone.marionette', options, &block)
def get_latest_version(opts)
get_npm_version('backbone.marionette', opts)
end
end
end

4
lib/docs/scrapers/markdown.rb
Unescape View File

@ -14,8 +14,8 @@ module Docs
Licensed under the BSD License.
HTML
def get_latest_version(options, &block)
block.call '1.0.0'
def get_latest_version(opts)
'1.0.0'
end
end
end

6
lib/docs/scrapers/matplotlib.rb
Unescape View File

@ -65,10 +65,8 @@ module Docs
]
end
def get_latest_version(options, &block)
get_latest_github_release('matplotlib', 'matplotlib', options) do |release|
block.call release['tag_name'][1..-1]
end
def get_latest_version(opts)
get_latest_github_release('matplotlib', 'matplotlib', opts)['tag_name'][1..-1]
end
end
end

7
lib/docs/scrapers/mdn/mdn.rb
Unescape View File

@ -21,10 +21,9 @@ module Docs
Licensed under the Creative Commons Attribution-ShareAlike License v2.5 or later.
HTML
def get_latest_version(opts, &block)
fetch_json("https://developer.mozilla.org/en-US/docs/feeds/json/tag/#{options[:mdn_tag]}", opts) do |json|
block.call DateTime.parse(json[0]['pubdate']).to_time.to_i
end
def get_latest_version(opts)
json = fetch_json("https://developer.mozilla.org/en-US/docs/feeds/json/tag/#{options[:mdn_tag]}", opts)
DateTime.parse(json[0]['pubdate']).to_time.to_i
end
private

7
lib/docs/scrapers/meteor.rb
Unescape View File

@ -46,10 +46,9 @@ module Docs
options[:fix_urls] = nil
end
def get_latest_version(options, &block)
fetch_doc('https://docs.meteor.com/#/full/', options) do |doc|
block.call doc.at_css('select.version-select > option').content
end
def get_latest_version(opts)
doc = fetch_doc('https://docs.meteor.com/#/full/', opts)
doc.at_css('select.version-select > option').content
end
end
end

4
lib/docs/scrapers/mocha.rb
Unescape View File

@ -19,8 +19,8 @@ module Docs
Licensed under the Creative Commons Attribution 4.0 International License.
HTML
def get_latest_version(options, &block)
get_npm_version('mocha', options, &block)
def get_latest_version(opts)
get_npm_version('mocha', opts)
end
end
end

4
lib/docs/scrapers/modernizr.rb
Unescape View File

@ -16,8 +16,8 @@ module Docs
Licensed under the MIT License.
HTML
def get_latest_version(options, &block)
get_npm_version('modernizr', options, &block)
def get_latest_version(opts)
get_npm_version('modernizr', opts)
end
end
end

7
lib/docs/scrapers/moment.rb
Unescape View File

@ -23,10 +23,9 @@ module Docs
Licensed under the MIT License.
HTML
def get_latest_version(options, &block)
fetch_doc('http://momentjs.com/', options) do |doc|
block.call doc.at_css('.hero-title > h1 > span').content
end
def get_latest_version(opts)
doc = fetch_doc('http://momentjs.com/', opts)
doc.at_css('.hero-title > h1 > span').content
end
end
end

9
lib/docs/scrapers/mongoose.rb
Unescape View File

@ -27,11 +27,10 @@ module Docs
Licensed under the MIT License.
HTML
def get_latest_version(options, &block)
fetch_doc('https://mongoosejs.com/docs/', options) do |doc|
label = doc.at_css('.pure-menu-link').content.strip
block.call label.sub(/Version /, '')
end
def get_latest_version(opts)
doc = fetch_doc('https://mongoosejs.com/docs/', opts)
label = doc.at_css('.pure-menu-link').content.strip
label.sub(/Version /, '')
end
end
end

9
lib/docs/scrapers/nginx.rb
Unescape View File

@ -26,11 +26,10 @@ module Docs
Licensed under the BSD License.
HTML
def get_latest_version(options, &block)
fetch_doc('https://nginx.org/en/download.html', options) do |doc|
table = doc.at_css('#content > table').inner_html
block.call table.scan(/nginx-([0-9.]+)</)[0][0]
end
def get_latest_version(opts)
doc = fetch_doc('https://nginx.org/en/download.html', opts)
table = doc.at_css('#content > table').inner_html
table.scan(/nginx-([0-9.]+)</)[0][0]
end
end
end

9
lib/docs/scrapers/nginx_lua_module.rb
Unescape View File

@ -19,11 +19,10 @@ module Docs
Licensed under the BSD License.
HTML
def get_latest_version(options, &block)
get_github_tags('openresty', 'lua-nginx-module', options) do |tags|
tag = tags.find { |tag| !tag['name'].include?('rc') }
block.call tag['name'][1..-1]
end
def get_latest_version(opts)
tags = get_github_tags('openresty', 'lua-nginx-module', opts)
tag = tags.find {|tag| !tag['name'].include?('rc')}
tag['name'][1..-1]
end
end
end

7
lib/docs/scrapers/nim.rb
Unescape View File

@ -18,10 +18,9 @@ module Docs
Licensed under the MIT License.
HTML
def get_latest_version(options, &block)
fetch_doc('https://nim-lang.org/docs/overview.html', options) do |doc|
block.call doc.at_css('.container > .docinfo > tbody > tr:last-child > td').content.strip
end
def get_latest_version(opts)
doc = fetch_doc('https://nim-lang.org/docs/overview.html', opts)
doc.at_css('.container > .docinfo > tbody > tr:last-child > td').content.strip
end
end
end

7
lib/docs/scrapers/node.rb
Unescape View File

@ -47,10 +47,9 @@ module Docs
self.base_url = 'https://nodejs.org/dist/latest-v4.x/docs/api/'
end
def get_latest_version(options, &block)
fetch_doc('https://nodejs.org/en/', options) do |doc|
block.call doc.at_css('#home-intro > .home-downloadblock:last-of-type > a')['data-version'][1..-1]
end
def get_latest_version(opts)
doc = fetch_doc('https://nodejs.org/en/', opts)
doc.at_css('#home-intro > .home-downloadblock:last-of-type > a')['data-version'][1..-1]
end
end
end

6
lib/docs/scrapers/nokogiri2.rb
Unescape View File

@ -20,10 +20,8 @@ module Docs
Licensed under the MIT License.
HTML
def get_latest_version(options, &block)
get_latest_github_release('sparklemotion', 'nokogiri', options) do |release|
block.call release['tag_name'][1..-1]
end
def get_latest_version(opts)
get_latest_github_release('sparklemotion', 'nokogiri', opts)['tag_name'][1..-1]
end
end
end

6
lib/docs/scrapers/npm.rb
Unescape View File

@ -30,10 +30,8 @@ module Docs
npm is a trademark of npm, Inc.
HTML
def get_latest_version(options, &block)
get_latest_github_release('npm', 'cli', options) do |release|
block.call release['tag_name'][1..-1]
end
def get_latest_version(opts)
get_latest_github_release('npm', 'cli', opts)['tag_name'][1..-1]
end
end
end

6
lib/docs/scrapers/numpy.rb
Unescape View File

@ -50,10 +50,8 @@ module Docs
self.base_url = "https://docs.scipy.org/doc/numpy-#{self.release}/reference/"
end
def get_latest_version(options, &block)
get_latest_github_release('numpy', 'numpy', options) do |release|
block.call release['tag_name'][1..-1]
end
def get_latest_version(opts)
get_latest_github_release('numpy', 'numpy', opts)['tag_name'][1..-1]
end
end
end

17
lib/docs/scrapers/openjdk.rb
Unescape View File

@ -87,7 +87,7 @@ module Docs
File.read(path).force_encoding('iso-8859-1').encode('utf-8') rescue nil
end
def get_latest_version(options, &block)
def get_latest_version(opts)
latest_version = 8
current_attempt = latest_version
attempts = 0
@ -95,17 +95,16 @@ module Docs
while attempts < 3
current_attempt += 1
fetch_doc("https://packages.debian.org/sid/openjdk-#{current_attempt}-doc", options) do |doc|
if doc.at_css('.perror').nil?
latest_version = current_attempt
attempts = 0
else
attempts += 1
end
doc = fetch_doc("https://packages.debian.org/sid/openjdk-#{current_attempt}-doc", opts)
if doc.at_css('.perror').nil?
latest_version = current_attempt
attempts = 0
else
attempts += 1
end
end
block.call latest_version
latest_version
end
end
end

6
lib/docs/scrapers/opentsdb.rb
Unescape View File

@ -19,10 +19,8 @@ module Docs
Licensed under the GNU LGPLv2.1+ and GPLv3+ licenses.
HTML
def get_latest_version(options, &block)
get_latest_github_release('OpenTSDB', 'opentsdb', options) do |release|
block.call release['tag_name'][1..-1]
end
def get_latest_version(opts)
get_latest_github_release('OpenTSDB', 'opentsdb', opts)['tag_name'][1..-1]
end
end
end

6
lib/docs/scrapers/padrino.rb
Unescape View File

@ -24,10 +24,8 @@ module Docs
request_one(url_for('index')).body
end
def get_latest_version(options, &block)
get_github_tags('padrino', 'padrino-framework', options) do |tags|
block.call tags[0]['name']
end
def get_latest_version(opts)
get_github_tags('padrino', 'padrino-framework', opts)[0]['name']
end
end
end

9
lib/docs/scrapers/pandas.rb
Unescape View File

@ -50,11 +50,10 @@ module Docs
self.base_url = "http://pandas.pydata.org/pandas-docs/version/#{self.release}/"
end
def get_latest_version(options, &block)
fetch_doc('http://pandas.pydata.org/pandas-docs/stable/', options) do |doc|
label = doc.at_css('.body > .section > p').content
block.call label.scan(/Version: ([0-9.]+)/)[0][0]
end
def get_latest_version(opts)
doc = fetch_doc('http://pandas.pydata.org/pandas-docs/stable/', opts)
label = doc.at_css('.body > .section > p').content
label.scan(/Version: ([0-9.]+)/)[0][0]
end
end
end

7
lib/docs/scrapers/perl.rb
Unescape View File

@ -44,10 +44,9 @@ module Docs
self.base_url = "https://perldoc.perl.org/#{self.release}/"
end
def get_latest_version(options, &block)
fetch('https://perldoc.perl.org/static/perlversion.js', options) do |body|
block.call body.scan(/>Perl ([0-9.]+)/)[0][0]
end
def get_latest_version(opts)
body = fetch('https://perldoc.perl.org/static/perlversion.js', opts)
body.scan(/>Perl ([0-9.]+)/)[0][0]
end
end
end

7
lib/docs/scrapers/phalcon.rb
Unescape View File

@ -30,10 +30,9 @@ module Docs
self.base_url = 'https://docs.phalconphp.com/en/2.0.0/'
end
def get_latest_version(options, &block)
fetch_doc('https://docs.phalconphp.com/', options) do |doc|
block.call doc.at_css('.custom-select__version').content.strip.sub(/Version /, '')
end
def get_latest_version(opts)
doc = fetch_doc('https://docs.phalconphp.com/', opts)
doc.at_css('.custom-select__version').content.strip.sub(/Version /, '')
end
end
end

6
lib/docs/scrapers/phaser.rb
Unescape View File

@ -26,10 +26,8 @@ module Docs
Licensed under the MIT License.
HTML
def get_latest_version(options, &block)
get_latest_github_release('photonstorm', 'phaser', options) do |release|
block.call release['tag_name'][1..-1]
end
def get_latest_version(opts)
get_latest_github_release('photonstorm', 'phaser', opts)['tag_name'][1..-1]
end
end
end

7
lib/docs/scrapers/phoenix.rb
Unescape View File

@ -47,10 +47,9 @@ module Docs
end
}
def get_latest_version(options, &block)
fetch_doc('https://hexdocs.pm/phoenix/Phoenix.html', options) do |doc|
block.call doc.at_css('.sidebar-projectVersion').content.strip[1..-1]
end
def get_latest_version(opts)
doc = fetch_doc('https://hexdocs.pm/phoenix/Phoenix.html', opts)
doc.at_css('.sidebar-projectVersion').content.strip[1..-1]
end
end
end

9
lib/docs/scrapers/php.rb
Unescape View File

@ -67,11 +67,10 @@ module Docs
Licensed under the Creative Commons Attribution License v3.0 or later.
HTML
def get_latest_version(options, &block)
fetch_doc('https://secure.php.net/manual/en/doc.changelog.php', options) do |doc|
label = doc.at_css('tbody.gen-changelog > tr > td').content
block.call label.split(',').last.strip
end
def get_latest_version(opts)
doc = fetch_doc('https://secure.php.net/manual/en/doc.changelog.php', opts)
label = doc.at_css('tbody.gen-changelog > tr > td').content
label.split(',').last.strip
end
end
end

Some files were not shown because too many files have changed in this diff Show More

Write Preview
Loading…
Cancel
Save