Clean up docs directory and fix links

pull/964/head
Jed Fox 6 years ago
parent 676f4f6e84
commit 73eab0bee5

@ -10,7 +10,26 @@ Keep track of development news:
* Watch the repository on [GitHub](https://github.com/freeCodeCamp/devdocs/subscription) * Watch the repository on [GitHub](https://github.com/freeCodeCamp/devdocs/subscription)
* Follow [@DevDocs](https://twitter.com/DevDocs) on Twitter * Follow [@DevDocs](https://twitter.com/DevDocs) on Twitter
**Table of Contents:** [Quick Start](#quick-start) · [Vision](#vision) · [App](#app) · [Scraper](#scraper) · [Commands](#available-commands) · [Contributing](#contributing) · [License](#copyright--license) · [Questions?](#questions) **Table of Contents:** [Plugins and Extensions](#plugins-and-extensions) • [Quick Start](#quick-start) · [Vision](#vision) · [App](#app) · [Scraper](#scraper) · [Commands](#available-commands) · [Contributing](#contributing) · [Documentation](#documentation) • [License](#copyright--license) · [Questions?](#questions)
## Plugins and Extensions
* [Chrome web app](https://chrome.google.com/webstore/detail/devdocs/mnfehgbmkapmjnhcnbodoamcioleeooe)
* [Ubuntu Touch app](https://uappexplorer.com/app/devdocsunofficial.berkes)
* [Sublime Text plugin](https://sublime.wbond.net/packages/DevDocs)
* [Atom plugin](https://atom.io/packages/devdocs)
* [Brackets extension](https://github.com/gruehle/dev-docs-viewer)
* [Fluid](http://fluidapp.com) for turning DevDocs into a real OS X app
* [GTK shell / Vim integration](https://github.com/naquad/devdocs-shell)
* [Emacs lookup](https://github.com/skeeto/devdocs-lookup)
* [Alfred Workflow](https://github.com/yannickglt/alfred-devdocs)
* [Vim search plugin with Devdocs in its defaults](https://github.com/waiting-for-dev/vim-www) Just set `let g:www_shortcut_engines = { 'devdocs': ['Devdocs', '<leader>dd'] }` to have a `:Devdocs` command and a `<leader>dd` mapping.
* [Visual Studio Code plugin](https://marketplace.visualstudio.com/items?itemName=akfish.vscode-devdocs ) (1)
* [Visual Studio Code plugin](https://marketplace.visualstudio.com/items?itemName=deibit.devdocs) (2)
* [Desktop application](https://github.com/egoist/devdocs-desktop)
* [Doc Browser](https://github.com/qwfy/doc-browser) is a native Linux app that supports DevDocs docsets
* [GNOME Application](https://github.com/hardpixel/devdocs-desktop) GTK3 application with search integrated in headerbar
## Quick Start ## Quick Start
@ -132,6 +151,13 @@ Contributions are welcome. Please read the [contributing guidelines](https://git
DevDocs's own documentation is available on the [wiki](https://github.com/freeCodeCamp/devdocs/wiki). DevDocs's own documentation is available on the [wiki](https://github.com/freeCodeCamp/devdocs/wiki).
## Documentation
* [Adding documentations to DevDocs](./docs/adding-docs.md)
* [Scraper Reference](./docs/scraper-reference.md)
* [Filter Reference](./docs/filter-reference.md)
* [Maintainers Guide](./docs/maintainers.md)
## Copyright / License ## Copyright / License
Copyright 2013-2018 Thibaut Courouble and [other contributors](https://github.com/freeCodeCamp/devdocs/graphs/contributors) Copyright 2013-2018 Thibaut Courouble and [other contributors](https://github.com/freeCodeCamp/devdocs/graphs/contributors)

@ -1,13 +1,11 @@
---
**Table of contents:** **Table of contents:**
* [Overview](https://github.com/Thibaut/devdocs/wiki/Filter-Reference#overview) * [Overview](#overview)
* [Instance methods](https://github.com/Thibaut/devdocs/wiki/Filter-Reference#instance-methods) * [Instance methods](#instance-methods)
* [Core filters](https://github.com/Thibaut/devdocs/wiki/Filter-Reference#core-filters) * [Core filters](#core-filters)
* [Custom filters](https://github.com/Thibaut/devdocs/wiki/Filter-Reference#custom-filters) * [Custom filters](#custom-filters)
- [CleanHtmlFilter](https://github.com/Thibaut/devdocs/wiki/Filter-Reference#cleanhtmlfilter) - [CleanHtmlFilter](#cleanhtmlfilter)
- [EntriesFilter](https://github.com/Thibaut/devdocs/wiki/Filter-Reference#entriesfilter) - [EntriesFilter](#entriesfilter)
## Overview ## Overview
@ -25,7 +23,7 @@ module Docs
end end
``` ```
Filters which manipulate the Nokogiri node object (`doc` and related methods) are _HTML filters_ and must not manipulate the HTML string (`html`). Vice-versa, filters which manipulate the string representation of the document are _text filters_ and must not manipulate the Nokogiri node object. The two types are divided into two stacks within the scrapers. These stacks are then combined into a pipeline that calls the HTML filters before the text filters (more details [here](https://github.com/Thibaut/devdocs/wiki/Scraper-Reference#filter-stacks)). This is to avoid parsing the document multiple times. Filters which manipulate the Nokogiri node object (`doc` and related methods) are _HTML filters_ and must not manipulate the HTML string (`html`). Vice-versa, filters which manipulate the string representation of the document are _text filters_ and must not manipulate the Nokogiri node object. The two types are divided into two stacks within the scrapers. These stacks are then combined into a pipeline that calls the HTML filters before the text filters (more details [here](./scraper-reference.md#filter-stacks)). This is to avoid parsing the document multiple times.
The `call` method must return either `doc` or `html`, depending on the type of filter. The `call` method must return either `doc` or `html`, depending on the type of filter.
@ -160,7 +158,7 @@ Each scraper must implement its own `EntriesFilter` by subclassing the [`Docs::E
* `include_default_entry?` [Boolean] * `include_default_entry?` [Boolean]
Whether to include the default entry. Whether to include the default entry.
Used when a page consists of multiple entries (returned by `additional_entries`) but doesn't have a name/type of its own, or to remove a page from the index (if it has no additional entries), in which case it won't be copied on the local filesystem and any link to it in the other pages will be broken (as explained on the [Scraper Reference](https://github.com/Thibaut/devdocs/wiki/Scraper-Reference) page, this is used to keep the `:skip` / `:skip_patterns` options to a maintainable size, or if the page includes links that can't reached from anywhere else). Used when a page consists of multiple entries (returned by `additional_entries`) but doesn't have a name/type of its own, or to remove a page from the index (if it has no additional entries), in which case it won't be copied on the local filesystem and any link to it in the other pages will be broken (as explained on the [Scraper Reference](./scraper-reference.md) page, this is used to keep the `:skip` / `:skip_patterns` options to a maintainable size, or if the page includes links that can't reached from anywhere else).
**Default:** `true` **Default:** `true`
* `additional_entries` [Array] * `additional_entries` [Array]

@ -1,23 +0,0 @@
### Development
* [Adding documentations to DevDocs](https://github.com/Thibaut/devdocs/wiki/Adding-documentations-to-DevDocs)
* [Scraper Reference](https://github.com/Thibaut/devdocs/wiki/Scraper-Reference)
* [Filter Reference](https://github.com/Thibaut/devdocs/wiki/Filter-Reference)
### Plugins and Extensions
* [Chrome web app](https://chrome.google.com/webstore/detail/devdocs/mnfehgbmkapmjnhcnbodoamcioleeooe)
* [Ubuntu Touch app](https://uappexplorer.com/app/devdocsunofficial.berkes)
* [Sublime Text plugin](https://sublime.wbond.net/packages/DevDocs)
* [Atom plugin](https://atom.io/packages/devdocs)
* [Brackets extension](https://github.com/gruehle/dev-docs-viewer)
* [Fluid](http://fluidapp.com) for turning DevDocs into a real OS X app
* [GTK shell / Vim integration](https://github.com/naquad/devdocs-shell)
* [Emacs lookup](https://github.com/skeeto/devdocs-lookup)
* [Alfred Workflow](https://github.com/yannickglt/alfred-devdocs)
* [Vim search plugin with Devdocs in its defaults](https://github.com/waiting-for-dev/vim-www) Just set `let g:www_shortcut_engines = { 'devdocs': ['Devdocs', '<leader>dd'] }` to have a `:Devdocs` command and a `<leader>dd` mapping.
* [Visual Studio Code plugin](https://marketplace.visualstudio.com/items?itemName=akfish.vscode-devdocs ) (1)
* [Visual Studio Code plugin](https://marketplace.visualstudio.com/items?itemName=deibit.devdocs) (2)
* [Desktop application](https://github.com/egoist/devdocs-desktop)
* [Doc Browser](https://github.com/qwfy/doc-browser) is a native Linux app that supports DevDocs docsets
* [GNOME Application](https://github.com/hardpixel/devdocs-desktop) GTK3 application with search integrated in headerbar

@ -1,12 +1,10 @@
---
**Table of contents:** **Table of contents:**
* [Overview](https://github.com/Thibaut/devdocs/wiki/Scraper-Reference#overview) * [Overview](#overview)
* [Configuration](https://github.com/Thibaut/devdocs/wiki/Scraper-Reference#configuration) * [Configuration](#configuration)
- [Attributes](https://github.com/Thibaut/devdocs/wiki/Scraper-Reference#attributes) - [Attributes](#attributes)
- [Filter stacks](https://github.com/Thibaut/devdocs/wiki/Scraper-Reference#filter-stacks) - [Filter stacks](#filter-stacks)
- [Filter options](https://github.com/Thibaut/devdocs/wiki/Scraper-Reference#filter-options) - [Filter options](#filter-options)
## Overview ## Overview
@ -34,9 +32,9 @@ Each URL is requested only once (case-insensitive).
Configuration is done via class attributes and divided into three main categories: Configuration is done via class attributes and divided into three main categories:
* [Attributes](https://github.com/Thibaut/devdocs/wiki/Scraper-Reference#attributes) — essential information such as name, version, URL, etc. * [Attributes](#attributes) — essential information such as name, version, URL, etc.
* [Filter stacks](https://github.com/Thibaut/devdocs/wiki/Scraper-Reference#filter-stacks) — the list of filters that will be applied to each page. * [Filter stacks](#filter-stacks) — the list of filters that will be applied to each page.
* [Filter options](https://github.com/Thibaut/devdocs/wiki/Scraper-Reference#filter-options) — the options passed to said filters. * [Filter options](#filter-options) — the options passed to said filters.
**Note:** scrapers are located in the [`lib/docs/scrapers`](https://github.com/Thibaut/devdocs/tree/master/lib/docs/scrapers/) directory. The class's name must be the [CamelCase](http://api.rubyonrails.org/classes/String.html#method-i-camelize) equivalent of the filename. **Note:** scrapers are located in the [`lib/docs/scrapers`](https://github.com/Thibaut/devdocs/tree/master/lib/docs/scrapers/) directory. The class's name must be the [CamelCase](http://api.rubyonrails.org/classes/String.html#method-i-camelize) equivalent of the filename.
@ -122,7 +120,7 @@ Additionally:
The filter options are stored in the `options` Hash. The Hash is inheritable (a recursive copy) and empty by default. The filter options are stored in the `options` Hash. The Hash is inheritable (a recursive copy) and empty by default.
More information about how filters work is available on the [Filter Reference](https://github.com/Thibaut/devdocs/wiki/Filter-Reference) page. More information about how filters work is available on the [Filter Reference](./filter-reference.md) page.
* [`ContainerFilter`](https://github.com/Thibaut/devdocs/blob/master/lib/docs/filters/core/container.rb) * [`ContainerFilter`](https://github.com/Thibaut/devdocs/blob/master/lib/docs/filters/core/container.rb)
@ -130,7 +128,7 @@ More information about how filters work is available on the [Filter Reference](h
A CSS selector of the container element. Everything outside of it will be removed and become unavailable to the other filters. If more than one element match the selector, the first one inside the DOM is used. If no elements match the selector, an error is raised. A CSS selector of the container element. Everything outside of it will be removed and become unavailable to the other filters. If more than one element match the selector, the first one inside the DOM is used. If no elements match the selector, an error is raised.
If the value is a Proc, it is called for each page with the filter instance as argument, and should return a selector or `nil`. If the value is a Proc, it is called for each page with the filter instance as argument, and should return a selector or `nil`.
The default container is the `<body>` element. The default container is the `<body>` element.
_Note: links outside of the container element will not be followed by the scraper. To remove links that should be followed, use a [`CleanHtml`](https://github.com/Thibaut/devdocs/wiki/Filter-Reference#cleanhtmlfilter) filter later in the stack._ _Note: links outside of the container element will not be followed by the scraper. To remove links that should be followed, use a [`CleanHtml`](./filter-reference.md#cleanhtmlfilter) filter later in the stack._
* [`NormalizeUrlsFilter`](https://github.com/Thibaut/devdocs/blob/master/lib/docs/filters/core/normalize_urls.rb) * [`NormalizeUrlsFilter`](https://github.com/Thibaut/devdocs/blob/master/lib/docs/filters/core/normalize_urls.rb)
The following options are used to modify URLs in the pages. They are useful to remove duplicates (when the same page is accessible from multiple URLs) and fix websites that have a bunch of redirections in place (when URLs that should be scraped, aren't, because they are behind a redirection which is outside of the `base_url` — see the MDN scrapers for examples of this). The following options are used to modify URLs in the pages. They are useful to remove duplicates (when the same page is accessible from multiple URLs) and fix websites that have a bunch of redirections in place (when URLs that should be scraped, aren't, because they are behind a redirection which is outside of the `base_url` — see the MDN scrapers for examples of this).
@ -170,7 +168,7 @@ More information about how filters work is available on the [Filter Reference](h
If the scraper has a `root_path`, the empty and `/` paths are automatically skipped. If the scraper has a `root_path`, the empty and `/` paths are automatically skipped.
If `:only` or `:only_patterns` is set, the root path is automatically added to `:only`. If `:only` or `:only_patterns` is set, the root path is automatically added to `:only`.
_Note: pages can be excluded from the index based on their content using the [`Entries`](https://github.com/Thibaut/devdocs/wiki/Filter-Reference#entriesfilter) filter. However, their URLs will still be converted to relative in the other pages and trying to open them will return a 404 error. Although not ideal, this is often better than having to maintain a long list of `:skip` URLs._ _Note: pages can be excluded from the index based on their content using the [`Entries`](./filter-reference.md#entriesfilter) filter. However, their URLs will still be converted to relative in the other pages and trying to open them will return a 404 error. Although not ideal, this is often better than having to maintain a long list of `:skip` URLs._
* [`AttributionFilter`](https://github.com/Thibaut/devdocs/blob/master/lib/docs/filters/core/attribution.rb) * [`AttributionFilter`](https://github.com/Thibaut/devdocs/blob/master/lib/docs/filters/core/attribution.rb)
@ -181,7 +179,7 @@ More information about how filters work is available on the [Filter Reference](h
- `:title` [String or Boolean or Proc] - `:title` [String or Boolean or Proc]
Unless the value is `false`, adds a title to every page. Unless the value is `false`, adds a title to every page.
If the value is `nil`, the title is the name of the page as determined by the [`Entries`](https://github.com/Thibaut/devdocs/wiki/Filter-Reference#entriesfilter) filter. Otherwise the title is the String or the value returned by the Proc (called for each page, with the filter instance as argument). If the Proc returns `nil` or `false`, no title is added. If the value is `nil`, the title is the name of the page as determined by the [`Entries`](./filter-reference.md#entriesfilter) filter. Otherwise the title is the String or the value returned by the Proc (called for each page, with the filter instance as argument). If the Proc returns `nil` or `false`, no title is added.
- `:root_title` [String or Boolean] - `:root_title` [String or Boolean]
Overrides the `:title` option for the root page only. Overrides the `:title` option for the root page only.

@ -1 +0,0 @@
Feel free to edit this page or ask for improvements on the [mailing list](https://groups.google.com/d/forum/devdocs).

@ -1,14 +1,14 @@
Adding a documentation may look like a daunting task but once you get the hang of it, it's actually quite simple. Don't hesitate to ask for help on the [mailing list](https://groups.google.com/d/forum/devdocs) if you ever get stuck. Adding a documentation may look like a daunting task but once you get the hang of it, it's actually quite simple. Don't hesitate to ask for help [in Gitter](https://gitter.im/FreeCodeCamp/DevDocs) if you ever get stuck.
**Note:** please read the [contributing guidelines](https://github.com/Thibaut/devdocs/blob/master/.github/CONTRIBUTING.md) before submitting a new documentation. **Note:** please read the [contributing guidelines](https://github.com/Thibaut/devdocs/blob/master/.github/CONTRIBUTING.md) before submitting a new documentation.
1. Create a subclass of `Docs::UrlScraper` or `Docs::FileScraper` in the `lib/docs/scrapers/` directory. Its name should be the [PascalCase](http://api.rubyonrails.org/classes/String.html#method-i-camelize) equivalent of the filename (e.g. `my_doc``MyDoc`) 1. Create a subclass of `Docs::UrlScraper` or `Docs::FileScraper` in the `lib/docs/scrapers/` directory. Its name should be the [PascalCase](http://api.rubyonrails.org/classes/String.html#method-i-camelize) equivalent of the filename (e.g. `my_doc``MyDoc`)
2. Add the appropriate class attributes and filter options (see the [Scraper Reference](https://github.com/Thibaut/devdocs/wiki/Scraper-Reference) page). 2. Add the appropriate class attributes and filter options (see the [Scraper Reference](./scraper-reference.md) page).
3. Check that the scraper is listed in `thor docs:list`. 3. Check that the scraper is listed in `thor docs:list`.
4. Create filters specific to the scraper in the `lib/docs/filters/[my_doc]/` directory and add them to the class's [filter stacks](https://github.com/Thibaut/devdocs/wiki/Scraper-Reference#filter-stacks). You may create any number of filters but will need at least the following two: 4. Create filters specific to the scraper in the `lib/docs/filters/[my_doc]/` directory and add them to the class's [filter stacks](./scraper-reference.md#filter-stacks). You may create any number of filters but will need at least the following two:
* A [`CleanHtml`](https://github.com/Thibaut/devdocs/wiki/Filter-Reference#cleanhtmlfilter) filter whose task is to clean the HTML markup (e.g. adding `id` attributes to headings) and remove everything superfluous and/or nonessential. * A [`CleanHtml`](./filter-reference.md#cleanhtmlfilter) filter whose task is to clean the HTML markup (e.g. adding `id` attributes to headings) and remove everything superfluous and/or nonessential.
* An [`Entries`](https://github.com/Thibaut/devdocs/wiki/Filter-Reference#entriesfilter) filter whose task is to determine the pages' metadata (the list of entries, each with a name, type and path). * An [`Entries`](./filter-reference.md#entriesfilter) filter whose task is to determine the pages' metadata (the list of entries, each with a name, type and path).
The [Filter Reference](https://github.com/Thibaut/devdocs/wiki/Filter-Reference) page has all the details about filters. The [Filter Reference](./filter-reference.md) page has all the details about filters.
5. Using the `thor docs:page [my_doc] [path]` command, check that the scraper works properly. Files will appear in the `public/docs/[my_doc]/` directory (but not inside the app as the command doesn't touch the index). `path` in this case refers to either the remote path (if using `UrlScraper`) or the local path (if using `FileScraper`). 5. Using the `thor docs:page [my_doc] [path]` command, check that the scraper works properly. Files will appear in the `public/docs/[my_doc]/` directory (but not inside the app as the command doesn't touch the index). `path` in this case refers to either the remote path (if using `UrlScraper`) or the local path (if using `FileScraper`).
6. Generate the full documentation using the `thor docs:generate [my_doc] --force` command. Additionally, you can use the `--verbose` option to see which files are being created/updated/deleted (useful to see what changed since the last run), and the `--debug` option to see which URLs are being requested and added to the queue (useful to pin down which page adds unwanted URLs to the queue). 6. Generate the full documentation using the `thor docs:generate [my_doc] --force` command. Additionally, you can use the `--verbose` option to see which files are being created/updated/deleted (useful to see what changed since the last run), and the `--debug` option to see which URLs are being requested and added to the queue (useful to pin down which page adds unwanted URLs to the queue).
7. Start the server, open the app, enable the documentation, and see how everything plays out. 7. Start the server, open the app, enable the documentation, and see how everything plays out.
Loading…
Cancel
Save