summaryrefslogtreecommitdiff
path: root/docs/dev
diff options
context:
space:
mode:
Diffstat (limited to 'docs/dev')
-rw-r--r--docs/dev/contribution_guide.rst59
-rw-r--r--docs/dev/engine_overview.rst101
-rw-r--r--docs/dev/index.rst3
-rw-r--r--docs/dev/makefile.rst257
-rw-r--r--docs/dev/plugins.rst15
-rw-r--r--docs/dev/quickstart.rst129
-rw-r--r--docs/dev/reST.rst68
-rw-r--r--docs/dev/search_api.rst8
8 files changed, 318 insertions, 322 deletions
diff --git a/docs/dev/contribution_guide.rst b/docs/dev/contribution_guide.rst
index 459dfb4..ed1c223 100644
--- a/docs/dev/contribution_guide.rst
+++ b/docs/dev/contribution_guide.rst
@@ -4,6 +4,11 @@
How to contribute
=================
+.. contents:: Contents
+ :depth: 2
+ :local:
+ :backlinks: entry
+
Prime directives: Privacy, Hackability
======================================
@@ -82,7 +87,7 @@ In order to submit a patch, please follow the steps below:
- Add yourself to the :origin:`AUTHORS.rst` file.
-- Choose meaning full commit messages, read `Conventional Commits`_
+- Choose meaningful commit messages, read `Conventional Commits`_
.. code::
@@ -112,8 +117,8 @@ Translation currently takes place on :ref:`transifex <translation>`.
Documentation
=============
-.. _Sphinx: http://www.sphinx-doc.org
-.. _reST: http://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html
+.. _Sphinx: https://www.sphinx-doc.org
+.. _reST: https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html
.. sidebar:: The reST sources
@@ -127,54 +132,58 @@ Here is an example which makes a complete rebuild:
.. code:: sh
- $ make docs-clean docs
+ $ make docs.clean docs.html
...
The HTML pages are in dist/docs.
-.. _make docs-live:
+.. _make docs.live:
live build
----------
-.. sidebar:: docs-clean
+.. _sphinx-autobuild:
+ https://github.com/executablebooks/sphinx-autobuild/blob/master/README.md
+
+.. sidebar:: docs.clean
It is recommended to assert a complete rebuild before deploying (use
- ``docs-clean``).
+ ``docs.clean``).
Live build is like WYSIWYG. If you want to edit the documentation, its
-recommended to use. The Makefile target ``docs-live`` builds the docs, opens
+recommended to use. The Makefile target ``docs.live`` builds the docs, opens
URL in your favorite browser and rebuilds every time a reST file has been
changed.
.. code:: sh
- $ make docs-live
+ $ make docs.live
...
The HTML pages are in dist/docs.
- ... Serving on http://0.0.0.0:8080
+ ... Serving on http://0.0.0.0:8000
... Start watching changes
+Live builds are implemented by sphinx-autobuild_. Use environment
+``$(SPHINXOPTS)`` to pass arguments to the sphinx-autobuild_ command. Except
+option ``--host`` (which is always set to ``0.0.0.0``) you can pass any
+argument. E.g to find and use a free port, use:
+
+.. code:: sh
+
+ $ SPHINXOPTS="--port 0" make docs.live
+ ...
+ ... Serving on http://0.0.0.0:50593
+ ...
+
.. _deploy on github.io:
deploy on github.io
-------------------
-To deploy documentation at :docs:`github.io <.>` use Makefile target
-:ref:`make gh-pages`, which will builds the documentation, clones searx into a sub
-folder ``gh-pages``, cleans it, copies the doc build into and runs all the
-needed git add, commit and push:
+To deploy documentation at :docs:`github.io <.>` use Makefile target :ref:`make
+docs.gh-pages`, which builds the documentation and runs all the needed git add,
+commit and push:
.. code:: sh
- $ make docs-clean gh-pages
- ...
- SPHINX docs --> file://<...>/dist/docs
- The HTML pages are in dist/docs.
- ...
- Cloning into 'gh-pages' ...
- ...
- cd gh-pages; git checkout gh-pages >/dev/null
- Switched to a new branch 'gh-pages'
- ...
- doc available at --> https://asciimoo.github.io/searx
+ $ make docs.clean docs.gh-pages
diff --git a/docs/dev/engine_overview.rst b/docs/dev/engine_overview.rst
index 449c837..bbd1413 100644
--- a/docs/dev/engine_overview.rst
+++ b/docs/dev/engine_overview.rst
@@ -18,6 +18,9 @@ engines. Adapters are stored under the folder :origin:`searx/engines`.
:depth: 3
:backlinks: entry
+
+.. _general engine configuration:
+
general engine configuration
============================
@@ -34,30 +37,36 @@ settings. However, the standard way is the following:
engine file
-----------
-======================= =========== ===========================================
+======================= =========== ========================================================
argument type information
-======================= =========== ===========================================
+======================= =========== ========================================================
categories list pages, in which the engine is working
paging boolean support multible pages
-language_support boolean support language choosing
time_range_support boolean support search time range
-offline boolean engine runs offline
-======================= =========== ===========================================
+engine_type str ``online`` by default, other possibles values are
+ ``offline``, ``online_dictionary``, ``online_currency``
+======================= =========== ========================================================
.. _engine settings:
settings.yml
------------
-======================= =========== ===========================================
+======================= =========== =============================================
argument type information
-======================= =========== ===========================================
+======================= =========== =============================================
name string name of search-engine
engine string name of searx-engine
(filename without ``.py``)
+enable_http bool enable HTTP
+ (by default only HTTPS is enabled).
shortcut string shortcut of search-engine
timeout string specific timeout for search-engine
-======================= =========== ===========================================
+display_error_messages boolean display error messages on the web UI
+proxies dict set proxies for a specific engine
+ (e.g. ``proxies : {http: socks5://proxy:port,
+ https: socks5://proxy:port}``)
+======================= =========== =============================================
overrides
@@ -89,7 +98,6 @@ example code
# engine dependent config
categories = ['general']
paging = True
- language_support = True
making a request
@@ -104,21 +112,48 @@ passed arguments
These arguments can be used to construct the search query. Furthermore,
parameters with default value can be redefined for special purposes.
+If the ``engine_type`` is ``online```:
+
+====================== ============== ========================================================================
+argument type default-value, information
+====================== ============== ========================================================================
+url str ``''``
+method str ``'GET'``
+headers set ``{}``
+data set ``{}``
+cookies set ``{}``
+verify bool ``True``
+headers.User-Agent str a random User-Agent
+category str current category, like ``'general'``
+safesearch int ``0``, between ``0`` and ``2`` (normal, moderate, strict)
+time_range Optional[str] ``None``, can be ``day``, ``week``, ``month``, ``year``
+pageno int current pagenumber
+language str specific language code like ``'en_US'``, or ``'all'`` if unspecified
+====================== ============== ========================================================================
+
+
+If the ``engine_type`` is ``online_dictionary```, in addition to the ``online`` arguments:
+
====================== ============ ========================================================================
argument type default-value, information
====================== ============ ========================================================================
-url string ``''``
-method string ``'GET'``
-headers set ``{}``
-data set ``{}``
-cookies set ``{}``
-verify boolean ``True``
-headers.User-Agent string a random User-Agent
-category string current category, like ``'general'``
-started datetime current date-time
-pageno int current pagenumber
-language string specific language code like ``'en_US'``, or ``'all'`` if unspecified
+from_lang str specific language code like ``'en_US'``
+to_lang str specific language code like ``'en_US'``
+query str the text query without the languages
+====================== ============ ========================================================================
+
+If the ``engine_type`` is ``online_currency```, in addition to the ``online`` arguments:
+
====================== ============ ========================================================================
+argument type default-value, information
+====================== ============ ========================================================================
+amount float the amount to convert
+from str ISO 4217 code
+to str ISO 4217 code
+from_name str currency name
+to_name str currency name
+====================== ============ ========================================================================
+
parsed arguments
----------------
@@ -127,16 +162,20 @@ The function ``def request(query, params):`` always returns the ``params``
variable. Inside searx, the following paramters can be used to specify a search
request:
-============ =========== =========================================================
-argument type information
-============ =========== =========================================================
-url string requested url
-method string HTTP request method
-headers set HTTP header information
-data set HTTP data information (parsed if ``method != 'GET'``)
-cookies set HTTP cookies
-verify boolean Performing SSL-Validity check
-============ =========== =========================================================
+=================== =========== ==========================================================================
+argument type information
+=================== =========== ==========================================================================
+url str requested url
+method str HTTP request method
+headers set HTTP header information
+data set HTTP data information
+cookies set HTTP cookies
+verify bool Performing SSL-Validity check
+follow_redirects bool Follow redirects
+max_redirects int maximum redirects, hard limit
+soft_max_redirects int maximum redirects, soft limit. Record an error but don't stop the engine
+raise_for_httperror bool True by default: raise an exception if the HTTP code of response is >= 300
+=================== =========== ==========================================================================
example code
@@ -255,7 +294,7 @@ latitude latitude of result (in decimal format)
longitude longitude of result (in decimal format)
boundingbox boundingbox of result (array of 4. values
``[lat-min, lat-max, lon-min, lon-max]``)
-geojson geojson of result (http://geojson.org)
+geojson geojson of result (https://geojson.org/)
osm.type type of osm-object (if OSM-Result)
osm.id id of osm-object (if OSM-Result)
address.name name of object
diff --git a/docs/dev/index.rst b/docs/dev/index.rst
index cb913a8..ba0a25a 100644
--- a/docs/dev/index.rst
+++ b/docs/dev/index.rst
@@ -3,7 +3,8 @@ Developer documentation
=======================
.. toctree::
- :maxdepth: 1
+ :maxdepth: 2
+ :caption: Contents
quickstart
contribution_guide
diff --git a/docs/dev/makefile.rst b/docs/dev/makefile.rst
index f595700..870b5d4 100644
--- a/docs/dev/makefile.rst
+++ b/docs/dev/makefile.rst
@@ -1,65 +1,33 @@
.. _makefile:
-================
-Makefile Targets
-================
+========
+Makefile
+========
.. _gnu-make: https://www.gnu.org/software/make/manual/make.html#Introduction
.. sidebar:: build environment
- Before looking deeper at the targets, first read about :ref:`makefile setup`
- and :ref:`make pyenv`.
+ Before looking deeper at the targets, first read about :ref:`make
+ install`.
-With the aim to simplify development cycles, started with :pull:`1756` a
-``Makefile`` based boilerplate was added. If you are not familiar with
-Makefiles, we recommend to read gnu-make_ introduction.
+ To install system requirements follow :ref:`buildhosts`.
+
+All relevant build tasks are implemented in :origin:`manage.sh` and for CI or
+IDE integration a small ``Makefile`` wrapper is available. If you are not
+familiar with Makefiles, we recommend to read gnu-make_ introduction.
The usage is simple, just type ``make {target-name}`` to *build* a target.
-Calling the ``help`` target gives a first overview::
-
- $ make help
- test - run developer tests
- docs - build documentation
- docs-live - autobuild HTML documentation while editing
- run - run developer instance
- install - developer install (./local)
- uninstall - uninstall (./local)
- gh-pages - build docs & deploy on gh-pages branch
- clean - drop builds and environments
- ...
+Calling the ``help`` target gives a first overview (``make help``):
+
+.. program-output:: bash -c "cd ..; make --no-print-directory help"
.. contents:: Contents
:depth: 2
:local:
:backlinks: entry
-
-.. _makefile setup:
-
-Setup
-=====
-
-.. _git stash: https://git-scm.com/docs/git-stash
-
-The main setup is done in the :origin:`Makefile`::
-
- export GIT_URL=https://github.com/asciimoo/searx
- export SEARX_URL=https://searx.me
- export DOCS_URL=https://asciimoo.github.io/searx
-
-.. sidebar:: fork & upstream
-
- Commit changes in your (local) branch, fork or whatever, but do not push them
- upstream / `git stash`_ is your friend.
-
-:GIT_URL: Changes this, to point to your searx fork.
-
-:SEARX_URL: Changes this, to point to your searx instance.
-
-:DOCS_URL: If you host your own (branded) documentation, change this URL.
-
-.. _make pyenv:
+.. _make install:
Python environment
==================
@@ -68,31 +36,42 @@ Python environment
``source ./local/py3/bin/activate``
-With Makefile we do no longer need to build up the virualenv manually (as
-described in the :ref:`devquickstart` guide). Jump into your git working tree
-and release a ``make pyenv``:
-
-.. code:: sh
+We do no longer need to build up the virtualenv manually. Jump into your git
+working tree and release a ``make install`` to get a virtualenv with a
+*developer install* of searx (:origin:`setup.py`). ::
$ cd ~/searx-clone
- $ make pyenv
- PYENV usage: source ./local/py3/bin/activate
+ $ make install
+ PYENV [virtualenv] installing ./requirements*.txt into local/py3
...
+ PYENV OK
+ PYENV [install] pip install -e 'searx[test]'
+ ...
+ Successfully installed argparse-1.4.0 searx
+ BUILDENV INFO:searx:load the default settings from ./searx/settings.yml
+ BUILDENV INFO:searx:Initialisation done
+ BUILDENV build utils/brand.env
-With target ``pyenv`` a development environment (aka virtualenv) was build up in
-``./local/py3/``. To make a *developer install* of searx (:origin:`setup.py`)
-into this environment, use make target ``install``:
-
-.. code:: sh
+If you release ``make install`` multiple times the installation will only
+rebuild if the sha256 sum of the *requirement files* fails. With other words:
+the check fails if you edit the requirements listed in
+:origin:`requirements-dev.txt` and :origin:`requirements.txt`). ::
$ make install
- PYENV usage: source ./local/py3/bin/activate
- PYENV using virtualenv from ./local/py3
- PYENV install .
-
-You have never to think about intermediate targets like ``pyenv`` or
-``install``, the ``Makefile`` chains them as requisites. Just run your main
-target.
+ PYENV OK
+ PYENV [virtualenv] requirements.sha256 failed
+ [virtualenv] - 6cea6eb6def9e14a18bf32f8a3e... ./requirements-dev.txt
+ [virtualenv] - 471efef6c73558e391c3adb35f4... ./requirements.txt
+ ...
+ PYENV [virtualenv] installing ./requirements*.txt into local/py3
+ ...
+ PYENV OK
+ PYENV [install] pip install -e 'searx[test]'
+ ...
+ Successfully installed argparse-1.4.0 searx
+ BUILDENV INFO:searx:load the default settings from ./searx/settings.yml
+ BUILDENV INFO:searx:Initialisation done
+ BUILDENV build utils/brand.env
.. sidebar:: drop environment
@@ -100,10 +79,7 @@ target.
<make clean>` first.
If you think, something goes wrong with your ./local environment or you change
-the :origin:`setup.py` file (or the requirements listed in
-:origin:`requirements-dev.txt` and :origin:`requirements.txt`), you have to call
-:ref:`make clean`.
-
+the :origin:`setup.py` file, you have to call :ref:`make clean`.
.. _make run:
@@ -113,88 +89,113 @@ the :origin:`setup.py` file (or the requirements listed in
To get up a running a developer instance simply call ``make run``. This enables
*debug* option in :origin:`searx/settings.yml`, starts a ``./searx/webapp.py``
instance, disables *debug* option again and opens the URL in your favorite WEB
-browser (:man:`xdg-open`):
+browser (:man:`xdg-open`)::
-.. code:: sh
-
- $ make run
- PYENV usage: source ./local/py3/bin/activate
- PYENV install .
- ./local/py3/bin/python ./searx/webapp.py
- ...
- INFO:werkzeug: * Running on http://127.0.0.1:8888/ (Press CTRL+C to quit)
- ...
+ $ make run
+ PYENV OK
+ SEARX_DEBUG=1 ./manage.sh pyenv.cmd python ./searx/webapp.py
+ ...
+ INFO:werkzeug: * Running on http://127.0.0.1:8888/ (Press CTRL+C to quit)
.. _make clean:
``make clean``
==============
-Drop all intermediate files, all builds, but keep sources untouched. Includes
-target ``pyclean`` which drops ./local environment. Before calling ``make
-clean`` stop all processes using :ref:`make pyenv`.
-
-.. code:: sh
+Drop all intermediate files, all builds, but keep sources untouched. Before
+calling ``make clean`` stop all processes using :ref:`make install`. ::
$ make clean
- CLEAN pyclean
- CLEAN clean
+ CLEAN pyenv
+ PYENV [virtualenv] drop ./local/py3
+ CLEAN docs -- ./build/docs ./dist/docs
+ CLEAN locally installed npm dependencies
+ CLEAN test stuff
+ CLEAN common files
.. _make docs:
-``make docs docs-live docs-clean``
-==================================
+``make docs docs.autobuild docs.clean``
+=======================================
-We describe the usage of the ``doc*`` targets in the :ref:`How to contribute /
+We describe the usage of the ``doc.*`` targets in the :ref:`How to contribute /
Documentation <contrib docs>` section. If you want to edit the documentation
-read our :ref:`make docs-live` section. If you are working in your own brand,
-adjust your :ref:`Makefile setup <makefile setup>`.
-
+read our :ref:`make docs.live` section. If you are working in your own brand,
+adjust your :ref:`settings global`.
-.. _make gh-pages:
+.. _make docs.gh-pages:
-``make gh-pages``
-=================
+``make docs.gh-pages``
+======================
-To deploy on github.io first adjust your :ref:`Makefile setup <makefile
-setup>`. For any further read :ref:`deploy on github.io`.
+To deploy on github.io first adjust your :ref:`settings global`. For any
+further read :ref:`deploy on github.io`.
.. _make test:
``make test``
=============
-Runs a series of tests: ``test.pep8``, ``test.unit``, ``test.robot`` and does
-additional :ref:`pylint checks <make pylint>`. You can run tests selective,
-e.g.:
+Runs a series of tests: :ref:`make test.pylint`, ``test.pep8``, ``test.unit``
+and ``test.robot``. You can run tests selective, e.g.::
-.. code:: sh
+ $ make test.pep8 test.unit test.sh
+ TEST test.pep8 OK
+ ...
+ TEST test.unit OK
+ ...
+ TEST test.sh OK
+
+.. _make test.sh:
- $ make test.pep8 test.unit
- . ./local/py3/bin/activate; ./manage.sh pep8_check
- [!] Running pep8 check
- . ./local/py3/bin/activate; ./manage.sh unit_tests
- [!] Running unit tests
+``make test.sh``
+================
+
+:ref:`sh lint` / if you have changed some bash scripting run this test before
+commit.
-.. _make pylint:
+.. _make test.pylint:
-``make pylint``
-===============
+``make test.pylint``
+====================
.. _Pylint: https://www.pylint.org/
-Before commiting its recommend to do some (more) linting. Pylint_ is known as
-one of the best source-code, bug and quality checker for the Python programming
-language. Pylint_ is not yet a quality gate within our searx project (like
-:ref:`test.pep8 <make test>` it is), but Pylint_ can help to improve code
-quality anyway. The pylint profile we use at searx project is found in
-project's root folder :origin:`.pylintrc`.
+Pylint_ is known as one of the best source-code, bug and quality checker for the
+Python programming language. The pylint profile we use at searx project is
+found in project's root folder :origin:`.pylintrc`.
+
+.. _make search.checker:
-Code quality is a ongoing process. Don't try to fix all messages from Pylint,
-run Pylint and check if your changed lines are bringing up new messages. If so,
-fix it. By this, code quality gets incremental better and if there comes the
-day, the linting is balanced out, we might decide to add Pylint as a quality
-gate.
+``search.checker.{engine name}``
+================================
+
+To check all engines::
+
+ make search.checker
+
+To check a engine with whitespace in the name like *google news* replace space
+by underline::
+
+ make search.checker.google_news
+
+To see HTTP requests and more use SEARX_DEBUG::
+
+ make SEARX_DEBUG=1 search.checker.google_news
+
+.. _3xx: https://en.wikipedia.org/wiki/List_of_HTTP_status_codes#3xx_redirection
+
+To filter out HTTP redirects (3xx_)::
+
+ make SEARX_DEBUG=1 search.checker.google_news | grep -A1 "HTTP/1.1\" 3[0-9][0-9]"
+ ...
+ Engine google news Checking
+ https://news.google.com:443 "GET /search?q=life&hl=en&lr=lang_en&ie=utf8&oe=utf8&ceid=US%3Aen&gl=US HTTP/1.1" 302 0
+ https://news.google.com:443 "GET /search?q=life&hl=en-US&lr=lang_en&ie=utf8&oe=utf8&ceid=US:en&gl=US HTTP/1.1" 200 None
+ --
+ https://news.google.com:443 "GET /search?q=computer&hl=en&lr=lang_en&ie=utf8&oe=utf8&ceid=US%3Aen&gl=US HTTP/1.1" 302 0
+ https://news.google.com:443 "GET /search?q=computer&hl=en-US&lr=lang_en&ie=utf8&oe=utf8&ceid=US:en&gl=US HTTP/1.1" 200 None
+ --
``make pybuild``
@@ -203,9 +204,7 @@ gate.
.. _PyPi: https://pypi.org/
.. _twine: https://twine.readthedocs.io/en/latest/
-Build Python packages in ``./dist/py``.
-
-.. code:: sh
+Build Python packages in ``./dist/py``::
$ make pybuild
...
@@ -213,9 +212,11 @@ Build Python packages in ``./dist/py``.
running sdist
running egg_info
...
- $ ls ./dist/py/
- searx-0.15.0-py3-none-any.whl searx-0.15.0.tar.gz
+ running bdist_wheel
+
+ $ ls ./dist
+ searx-0.18.0-py3-none-any.whl searx-0.18.0.tar.gz
-To upload packages to PyPi_, there is also a ``upload-pypi`` target. It needs
-twine_ to be installed. Since you are not the owner of :pypi:`searx` you will
-never need the latter.
+To upload packages to PyPi_, there is also a ``pypi.upload`` target (to test use
+``pypi.upload.test``). Since you are not the owner of :pypi:`searx` you will
+never need to upload.
diff --git a/docs/dev/plugins.rst b/docs/dev/plugins.rst
index 2bf44f1..16262ea 100644
--- a/docs/dev/plugins.rst
+++ b/docs/dev/plugins.rst
@@ -30,6 +30,21 @@ Example plugin
ctx['search'].suggestions.add('example')
return True
+External plugins
+================
+
+External plugins are standard python modules implementing all the requirements of the standard plugins.
+Plugins can be enabled by adding them to :ref:`settings.yml`'s ``plugins`` section.
+Example external plugin can be found `here <https://github.com/asciimoo/searx_external_plugin_example>`_.
+
+Register your plugin
+====================
+
+To enable your plugin register your plugin in
+searx > plugin > __init__.py.
+And at the bottom of the file add your plugin like.
+``plugins.register(name_of_python_file)``
+
Plugin entry points
===================
diff --git a/docs/dev/quickstart.rst b/docs/dev/quickstart.rst
index e40772b..bb9f4d6 100644
--- a/docs/dev/quickstart.rst
+++ b/docs/dev/quickstart.rst
@@ -4,129 +4,52 @@
Development Quickstart
======================
-.. sidebar:: :ref:`makefile`
+.. _npm: https://www.npmjs.com/
- For additional developer purpose there are :ref:`makefile`.
-
-This quickstart guide gets your environment set up with searx. Furthermore, it
-gives a short introduction to the ``manage.sh`` script.
-
-How to setup your development environment
-=========================================
-
-.. sidebar:: :ref:`make pyenv <make pyenv>`
-
- Alternatively use the :ref:`make pyenv`.
-
-First, clone the source code of searx to the desired folder. In this case the
-source is cloned to ``~/myprojects/searx``. Then create and activate the
-searx-ve virtualenv and install the required packages using ``manage.sh``.
-
-.. code:: sh
-
- cd ~/myprojects
- git clone https://github.com/asciimoo/searx.git
- cd searx
- virtualenv searx-ve
- . ./searx-ve/bin/activate
- ./manage.sh update_dev_packages
-
-
-How to run tests
-================
-
-.. sidebar:: :ref:`make test.unit <make test>`
-
- Alternatively use the ``test.pep8``, ``test.unit``, ``test.robot`` targets.
-
-Tests can be run using the ``manage.sh`` script. Following tests and checks are
-available:
-
-- Unit tests
-- Selenium tests
-- PEP8 validation
-- Unit test coverage check
-
-For example unit tests are run with the command below:
-
-.. code:: sh
-
- ./manage.sh unit_tests
-
-For further test options, please consult the help of the ``manage.sh`` script or
-read :ref:`make test`.
-
-
-How to compile styles and javascript
-====================================
-
-.. _less: http://lesscss.org/
-.. _NodeJS: https://nodejs.org
-
-How to build styles
--------------------
-
-Less_ is required to build the styles of searx. Less_ can be installed using
-either NodeJS_ or Apt.
+Searx loves developers, just clone and start hacking. All the rest is done for
+you simply by using :ref:`make <makefile>`.
.. code:: sh
- sudo -H apt-get install nodejs
- sudo -H npm install -g less
+ git clone https://github.com/searx/searx.git
-OR
+Here is how a minimal workflow looks like:
-.. code:: sh
+1. *start* hacking
+2. *run* your code: :ref:`make run`
+3. *test* your code: :ref:`make test`
- sudo -H apt-get install node-less
+If you think at some point something fails, go back to *start*. Otherwise,
+choose a meaningful commit message and we are happy to receive your pull
+request. To not end in *wild west* we have some directives, please pay attention
+to our ":ref:`how to contribute`" guideline.
-After satisfying the requirements styles can be build using ``manage.sh``
+If you implement themes, you will need to compile styles and JavaScript before
+*run*.
.. code:: sh
- ./manage.sh styles
-
-
-How to build the source of the oscar theme
-==========================================
-
-.. _grunt: https://gruntjs.com/
+ make themes
-Grunt_ must be installed in order to build the javascript sources. It depends on
-NodeJS, so first Node has to be installed.
+Don't forget to install npm_ first.
-.. code:: sh
-
- sudo -H apt-get install nodejs
- sudo -H npm install -g grunt-cli
+.. tabs::
-After installing grunt, the files can be built using the following command:
-
-.. code:: sh
+ .. group-tab:: Ubuntu / debian
- ./manage.sh grunt_build
+ .. code:: sh
+ sudo -H apt-get install npm
-Tips for debugging/development
-==============================
+ .. group-tab:: Arch Linux
-.. sidebar:: :ref:`make run`
+ .. code-block:: sh
- Makefile target ``run`` already enables debug option for your developer
- session / see :ref:`make run`.
+ sudo -H pacman -S npm
-Turn on debug logging
- Whether you are working on a new engine or trying to eliminate a bug, it is
- always a good idea to turn on debug logging. When debug logging is enabled a
- stack trace appears, instead of the cryptic ``Internal Server Error``
- message. It can be turned on by setting ``debug: False`` to ``debug: True`` in
- :origin:`settings.yml <searx/settings.yml>`.
+ .. group-tab:: Fedora / RHEL
-.. sidebar:: :ref:`make test`
+ .. code-block:: sh
- Alternatively use the :ref:`make test` targets.
+ sudo -H dnf install npm
-Run ``./manage.sh tests`` before creating a PR.
- Failing build on Travis is common because of PEP8 checks. So a new commit
- must be created containing these format fixes. This phase can be skipped if
- ``./manage.sh tests`` is run locally before creating a PR.
diff --git a/docs/dev/reST.rst b/docs/dev/reST.rst
index 4dc1279..1817504 100644
--- a/docs/dev/reST.rst
+++ b/docs/dev/reST.rst
@@ -15,8 +15,8 @@ generated and deployed at :docs:`github.io <.>`. For build prerequisites read
:ref:`docs build`.
The source files of Searx's documentation are located at :origin:`docs`. Sphinx
-assumes source files to be encoded in UTF-8 by defaul. Run :ref:`make docs-live
-<make docs-live>` to build HTML while editing.
+assumes source files to be encoded in UTF-8 by defaul. Run :ref:`make docs.live
+<make docs.live>` to build HTML while editing.
.. sidebar:: Further reading
@@ -285,7 +285,7 @@ content becomes smart.
:rst:role:`pep` :pep:`8` ``:pep:`8```
sphinx.ext.extlinks_
--------------------------------------------------------------------------------------------------
- project's wiki article :wiki:`Searx-instances` ``:wiki:`Searx-instances```
+ project's wiki article :wiki:`Offline-engines` ``:wiki:`Offline-engines```
to docs public URL :docs:`dev/reST.html` ``:docs:`dev/reST.html```
files & folders origin :origin:`docs/dev/reST.rst` ``:origin:`docs/dev/reST.rst```
pull request :pull:`1756` ``:pull:`1756```
@@ -319,14 +319,18 @@ To list all anchors of the inventory (e.g. ``python``) use:
.. code:: sh
$ python -m sphinx.ext.intersphinx https://docs.python.org/3/objects.inv
+ ...
+ $ python -m sphinx.ext.intersphinx https://searx.github.io/searx/objects.inv
+ ...
Literal blocks
==============
The simplest form of :duref:`literal-blocks` is a indented block introduced by
two colons (``::``). For highlighting use :dudir:`highlight` or :ref:`reST
-code` directive. To include literals from external files use directive
-:dudir:`literalinclude`.
+code` directive. To include literals from external files use
+:rst:dir:`literalinclude` or :ref:`kernel-include <kernel-include-directive>`
+directive (latter one expands environment variables in the path name).
.. _reST literal:
@@ -1272,28 +1276,33 @@ Templating
.. sidebar:: Build environment
- All *generic-doc* tasks are running in the :ref:`build environment <make
- pyenv>`.
+ All *generic-doc* tasks are running in the :ref:`make install`.
Templating is suitable for documentation which is created generic at the build
-time. The sphinx-jinja_ extension evaluates jinja_ templates in the :ref:`build
-environment <make pyenv>` (with searx modules installed). We use this e.g. to
-build chapter: :ref:`engines generic`. Below the jinja directive from the
+time. The sphinx-jinja_ extension evaluates jinja_ templates in the :ref:`make
+install` (with searx modules installed). We use this e.g. to build chapter:
+:ref:`engines generic`. Below the jinja directive from the
:origin:`docs/admin/engines.rst` is shown:
.. literalinclude:: ../admin/engines.rst
:language: reST
:start-after: .. _configured engines:
-The context for the template is selected in the line ``.. jinja:: webapp``. In
-sphinx's build configuration (:origin:`docs/conf.py`) the ``webapp`` context
-points to the name space of the python module: ``webapp``.
+The context for the template is selected in the line ``.. jinja:: searx``. In
+sphinx's build configuration (:origin:`docs/conf.py`) the ``searx`` context
+contains the ``engines`` and ``plugins``.
.. code:: py
- from searx import webapp
+ import searx.search
+ import searx.engines
+ import searx.plugins
+ searx.search.initialize()
jinja_contexts = {
- 'webapp': dict(**webapp.__dict__)
+ 'searx': {
+ 'engines': searx.engines.engines,
+ 'plugins': searx.plugins.plugins
+ },
}
@@ -1312,9 +1321,8 @@ others are basic-tabs_ and code-tabs_. Below a *group-tab* example from
.. literalinclude:: ../admin/buildhosts.rst
:language: reST
- :start-after: .. _system requirements:
- :end-before: .. _system requirements END:
-
+ :start-after: .. SNIP sh lint requirements
+ :end-before: .. SNAP sh lint requirements
.. _math:
@@ -1391,27 +1399,27 @@ The next example shows the difference of ``\tfrac`` (*textstyle*) and ``\dfrac``
.. _readability: https://docs.python-guide.org/writing/style/
.. _Sphinx-Primer:
- http://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html
+ https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html
.. _reST: https://docutils.sourceforge.io/rst.html
.. _Sphinx Roles:
https://www.sphinx-doc.org/en/master/usage/restructuredtext/roles.html
-.. _Sphinx: http://www.sphinx-doc.org
-.. _`sphinx-doc FAQ`: http://www.sphinx-doc.org/en/stable/faq.html
+.. _Sphinx: https://www.sphinx-doc.org
+.. _`sphinx-doc FAQ`: https://www.sphinx-doc.org/en/stable/faq.html
.. _Sphinx markup constructs:
- http://www.sphinx-doc.org/en/stable/markup/index.html
+ https://www.sphinx-doc.org/en/stable/markup/index.html
.. _`sphinx cross references`:
- http://www.sphinx-doc.org/en/stable/markup/inline.html#cross-referencing-arbitrary-locations
+ https://www.sphinx-doc.org/en/stable/markup/inline.html#cross-referencing-arbitrary-locations
.. _sphinx.ext.extlinks:
https://www.sphinx-doc.org/en/master/usage/extensions/extlinks.html
-.. _intersphinx: http://www.sphinx-doc.org/en/stable/ext/intersphinx.html
-.. _sphinx config: http://www.sphinx-doc.org/en/stable/config.html
-.. _Sphinx's autodoc: http://www.sphinx-doc.org/en/stable/ext/autodoc.html
+.. _intersphinx: https://www.sphinx-doc.org/en/stable/ext/intersphinx.html
+.. _sphinx config: https://www.sphinx-doc.org/en/stable/config.html
+.. _Sphinx's autodoc: https://www.sphinx-doc.org/en/stable/ext/autodoc.html
.. _Sphinx's Python domain:
- http://www.sphinx-doc.org/en/stable/domains.html#the-python-domain
+ https://www.sphinx-doc.org/en/stable/domains.html#the-python-domain
.. _Sphinx's C domain:
- http://www.sphinx-doc.org/en/stable/domains.html#cross-referencing-c-constructs
+ https://www.sphinx-doc.org/en/stable/domains.html#cross-referencing-c-constructs
.. _doctree:
- http://www.sphinx-doc.org/en/master/extdev/tutorial.html?highlight=doctree#build-phases
+ https://www.sphinx-doc.org/en/master/extdev/tutorial.html?highlight=doctree#build-phases
.. _docutils: http://docutils.sourceforge.net/docs/index.html
.. _docutils FAQ: http://docutils.sourceforge.net/FAQ.html
.. _linuxdoc: https://return42.github.io/linuxdoc
@@ -1424,5 +1432,5 @@ The next example shows the difference of ``\tfrac`` (*textstyle*) and ``\dfrac``
.. _ImageMagick: https://www.imagemagick.org
.. _`Emacs Table Mode`: https://www.emacswiki.org/emacs/TableMode
-.. _`Online Tables Generator`: http://www.tablesgenerator.com/text_tables
+.. _`Online Tables Generator`: https://www.tablesgenerator.com/text_tables
.. _`OASIS XML Exchange Table Model`: https://www.oasis-open.org/specs/tm9901.html
diff --git a/docs/dev/search_api.rst b/docs/dev/search_api.rst
index 922548f..68fee94 100644
--- a/docs/dev/search_api.rst
+++ b/docs/dev/search_api.rst
@@ -6,7 +6,7 @@ Search API
The search supports both ``GET`` and ``POST``.
-Furthermore, two enpoints ``/`` and ``/search`` are available for querying.
+Furthermore, two endpoints ``/`` and ``/search`` are available for querying.
``GET /``
@@ -76,12 +76,12 @@ Parameters
supports safe search in the preferences page of an instance.
``theme`` : default ``oscar``
- [ ``oscar``, ``simple``, ``legacy``, ``pix-art``, ``courgette`` ]
+ [ ``oscar``, ``simple`` ]
Theme of instance.
Please note, available themes depend on an instance. It is possible that an
- instance administrator deleted, created or renamed themes on his/her instance.
+ instance administrator deleted, created or renamed themes on their instance.
See the available options in the preferences page of the instance.
``oscar-style`` : default ``logicodev``
@@ -91,7 +91,7 @@ Parameters
``oscar``.
Please note, available styles depend on an instance. It is possible that an
- instance administrator deleted, created or renamed styles on his/her
+ instance administrator deleted, created or renamed styles on their
instance. See the available options in the preferences page of the instance.
``enabled_plugins`` : optional
generated by cgit v1.2.3 (git 2.25.1) at 2024-05-01 09:24:36 +0000