===========
Maintenance
===========

JSON Schema Repo
================

pydvid includes the `service-contracts`_ repo as a `git subtree` located at prefix pydvid/dvidschema.  
Older versionos of git do not include the ``subtree`` command, so you may need to upgrade git and/or 
manually install the ``subtree`` command into your git installation.

Changes to the dvidschema repo must be pulled into pydvid using ``git subtree pull``.  For example:

.. code-block:: bash

    # Add the dvidschema repo as a remote if you haven't yet...
    git remote add -f janelia-service-contracts https://github.com/janelia-flyem/service-contracts
    
    # Pull the most recent changes from service-contracts into our own repo's subtree
    git fetch janelia-service-contracts
    git subtree pull --prefix pydvid/dvidschemas janelia-service-contracts master

.. _service-contracts: https://github.com/janelia-flyem/service-contracts


Testing
=======

The test suite requires the following additional dependencies:

* `nosetests <https://nose.readthedocs.org/en/latest/>`_
* `h5py <http://www.h5py.org/>`_

Here's how to run the test suite:

.. code-block:: bash

    cd tests
    PYTHONPATH=.. nosetests .

Note that pydvid uses its own "mock server" for testing purposes, which mimics the real responses provided by the DVID server.
This means that you cannot have DVID running while running the test suite, as it would conflict with the mock server in the test suite.

Mock Server
-----------
For test purposes, a mock DVID server is implemented in the ``mockserver`` directory.
It serves up HDF5 datasets over http using the `DVID REST API`_.

.. _DVID REST API: http://godoc.org/github.com/janelia-flyem/dvid/datatype/voxels#pkg-constants

The mock server pulls its data from an hdf5 file with a special structure.
The ``H5MockServerDataFile`` utility class can be used to generate the file:

.. code-block:: python

    import numpy
    from mockserver.h5mockserver import H5MockServerDataFile
    
    # Generate a volume to store.
    data = numpy.random.randint( 0, 256, (1,100,200,300) )
    voxels_metadata = VoxelsMetadata.create_default_metadata( (1,100,200,300), numpy.uint8, 'cxyz', 1.0, "" )
    
    # Create special server datafile with one dataset, with one node.
    # Then add our data volume to it.
    with H5MockServerDataFile( 'mock_storage.h5' ) as server_datafile:
        server_datafile.add_node( 'my_dataset', 'abc123' )
        server_datafile.add_volume( 'my_dataset', 'my_volume', data_view, voxels_metadata )
    
Once you have a datafile, the server can be started from the command line:

.. code-block:: bash
    
    $ cd mockserver
    $ PYTHONPATH=.. python h5mockserver.py mock_storage.h5
    
See ``h5mockserver.py`` the datafile format details.
    
Maintaining this Documentation
==============================

This documentation is built with `Sphinx <http://sphinx-doc.org/>`_, and 
hosted on github as a `github page <http://pages.github.com/>`_ for the pydvid repo.

To make a changes to the docs, edit the desired ``.rst`` file(s) in the docs directory, and then build the docs:

.. code-block:: bash

    cd docs
    PYTHONPATH=.. make html

First, view your changes locally:

.. code-block:: bash

    firefox build/html/index.html

Your changes will not be visible online until they are applied to the special ``gh-pages`` branch of pydvid and pushed.

Fortunately, there is a script in pydvid for automating this process.

.. note:: To avoid losing any uncommitted changes in your working copy, it is highly recommended that you use a special local copy of the pydvid repo to do this.

Just follow these steps:

1) Make sure your changes to the .rst files are pushed to pydvid/master.
2) Make a new clone of the pydvid repo, and checkout the ``gh-pages`` branch.
3) Run the ``update_from_master.sh`` script.

Here's a walk-through (output not shown).

.. code-block:: bash

    # 1) Commit your documentation changes, and push them to origin/master
    cd docs
    git add -u .
    git commit -m "Made some docs changes."
    git push origin master

    # 2a) Make a NEW CLONE of the repo (leave your working copy alone.)
    cd /tmp
    git clone ssh://git@github.com/ilastik/pydvid pydvid-gh-pages
    
    # 2b) Checkout the gh-pages branch
    cd pydvid-gh-pages/
    git checkout gh-pages

    # Run update_from_master.sh to make your changes visible on the web.
    ./update_from_master.sh 

The ``update_from_master.sh`` script handles the necessary pre-processing required by the github pages system.
You can view the updated documentation at `<http://janelia-flyem.github.com/pydvid>`_.