Commit Graph

121 Commits (eda53194b029c3bed977d09892966e1bfd5b45b9)

Author SHA1 Message Date
Simon Legner c25fbb4b3c doc.fetch: read GITHUB_TOKEN also from ENV
5 years ago
Jasper van Merle c88ae2c1e1 reactivex: implement get_latest_version
6 years ago
Jasper van Merle 6e917001ce Fix merge conflicts
6 years ago
Jasper van Merle 8325ed78ef
Fix OpenJDK scraper
6 years ago
Jasper van Merle 7084d9f082 Improve get_latest_github_release
6 years ago
Jasper van Merle 3cb97f71b3
fetching -> connecting to
6 years ago
Jasper van Merle 11eeadbc0a Better error handling and added a 15s timeout on HTTP requests
6 years ago
Jasper van Merle 41e0a138a6 Refactoring and cleaning up
6 years ago
Jasper van Merle cd43632a2c Check for updates based on modified date for scrapers without release
6 years ago
Jasper van Merle 3dc17a9b29 Finish get_latest_version for 81 scrapers and add uploading functionality
6 years ago
Jasper van Merle 2d1e8aa00c Add better logging and get_latest_version implementations for 10 scrapers
6 years ago
Jasper van Merle d0300dd585 Create command to check for updates
6 years ago
Thibaut Courouble 2eb865442c Bump Ruby 2.6.0
6 years ago
Thibaut Courouble bf003669ba Simplify file scraper setup; scrape files in the "docs/[slug]" directory
6 years ago
Thibaut Courouble 9e37ff5c8a Add instrumentation for slow response processing
6 years ago
Thibaut Courouble c7a10efe5b Add ParseCfEmail filter to the default stack used by all scrapers
7 years ago
Jed Fox 608853bd2e Convert the `decode_cloudflare_email` helper to a dedicated filter class
7 years ago
Thibaut Courouble 68b80bce36 Generate pretty JSON in docs.json manifest
7 years ago
Thibaut Courouble 0095779b3f Instrument FileScraper read errors rather than use puts
7 years ago
Thibaut Courouble 0725a69af5 Store docs' metadata in meta.json files
7 years ago
Thibaut Courouble e3ef139111 Fix path encoding issues in C/C++ scraper
7 years ago
Thibaut Courouble 472f03d5b0 Update C/C++ documentation
7 years ago
Thibaut Courouble ad9276e901 Add rate limiter
7 years ago
Thibaut Courouble 914100a657 Improve error message on failed requests
7 years ago
Thibaut Courouble 3f43c03dbc Update Angular documentation (4.3.2)
8 years ago
Thibaut Courouble 65e8236ab5 Switch to headless Chrome for capybara/selenium scraping
8 years ago
Thibaut Courouble ec3bb9169d Properly disable http pipelining
8 years ago
Thibaut Courouble f7580a8b52 Download images by default
8 years ago
Thibaut Courouble a6855329e8 Add image scraping and optimization filter
8 years ago
Thibaut Courouble 6fc48db8af Improve error logging
8 years ago
Thibaut Courouble 8ef3502cb3 Throw an error when response code == 0
8 years ago
Thibaut Courouble ee21c0bf29 Add frozen_string_literal to subscriber files"
8 years ago
Thibaut Courouble 2efce74521 Finish OpenJDK scraper
8 years ago
Thibaut Courouble 057a3a9ce9 Ensure root page is processed first for docs with multiple base urls
8 years ago
Thibaut Courouble 509fb443e6 Remove leftover debug log
8 years ago
Thibaut Courouble c1ebb7a0b9 Improve Doc#name and Doc#slug
8 years ago
Andreas Stenius b36f3f8095 core/doc: make sure name is usable as slug.
8 years ago
Thibaut Courouble 476c69e419 Improve MDN scrapers
8 years ago
Thibaut Courouble 8ddb812035 Pass response object to Scraper#parse method
8 years ago
Thibaut Courouble 4e41ed9f25 Add <base> support
8 years ago
Thibaut Courouble 6f0214eaf3 Make Docs::Parser return the entire document instead of <body>
8 years ago
Thibaut Courouble 8147f9f90a Update TensorFlow documentation (0.12)
8 years ago
Thibaut Courouble 0c8ca4e5fa Add SQLite documentation
8 years ago
Thibaut Courouble 0d4c6b51bb Update TensorFlow documentation (0.11)
8 years ago
Thibaut Courouble 721adf8e21 Don't rewrite data URIs
8 years ago
Thibaut Courouble 6fdf81e649 Update Elixir documentation (1.3.3)
8 years ago
Thibaut Courouble 46a9ed16f6 Refactor scrapers with multiple base URLs
8 years ago
Thibaut Courouble 64eb1c86fc Log errors instead of exiting when scraping docs
9 years ago
Thibaut Courouble 82d0725747 Improve ordering of entries and types
9 years ago
Thibaut Courouble b279f84a4c Update and version Meteor documentation
9 years ago