Skip to content

automatically deploy development docs #386

Closed
Closed
automatically deploy development docs#386
Assignees
orbeckst
Labels
Component-DocsContinuous IntegrationPriority-Lowtesting
Milestone
0.13

Description

@orbeckst
orbeckst
opened on Aug 7, 2015

Following Automatically Update Github Pages with Travis Example seems straighforward enough in order to have travis-ci build the sphinx docs and then push them to the gh-pages branch. (See also the discussion at the bottom of #350.)

Potential solutions

Progress so far

  • generated OAUTH token (limit scope to public_repo access; I used the tokens web interface but can also be done via curl as described in the awestruct gh deployment notes). DONE

  • encrypt for travis with the travis client:

    gem2.0 install travis
travis encrypt GH_TOKEN=`cat ./github_OAUTH_token`

    yields the entry for the .travis.yml file:

     secure: "HIj3p+p2PV8DBVg/KGUx6n83KwB0ASE5FwOn0SMB9zxnzAqe8sapwdBQdMdq0sXB7xT1spJqRxuxOMVEVn35BNLu7bxMLfa4287C8YXcomnvmv9xruxAsjsIewnNQ80vtPVbQddBPxa4jKbqgPby5QhhAP8KANAqYe44pIV70fY="

    (This is encrypted with the travis public key. For more details, see Travis CI: Encryption keys.)

  • add lines to .travis.yml along the lines of

    script:
  - ./testsuite/MDAnalysisTests/mda_nosetests -v --with-coverage --cover-package MDAnalysis --processes=2 --process-timeout=120 --with-memleak
  - coveralls
  - test $TRAVIS_PULL_REQUEST == "false" && test $TRAVIS_BRANCH == "develop" && cd package/doc/sphinx && make clean html

after_success:
  - test $TRAVIS_PULL_REQUEST == "false" && test $TRAVIS_BRANCH == "develop" && bash maintainer/deploy.sh

env:
   global:
      secure: "HIj3p+p2PV8DBVg/KGUx6n83KwB0ASE5FwOn0SMB9zxnzAqe8sapwdBQdMdq0sXB7xT1spJqRxuxOMVEVn35BNLu7bxMLfa4287C8YXcomnvmv9xruxAsjsIewnNQ80vtPVbQddBPxa4jKbqgPby5QhhAP8KANAqYe44pIV70fY="

    (We want to build the developer docs when we push an update to the developer branch.)

    I don't know if an error during doc building will then also signal a failure of a build. Perhaps doc building should be put in a sub shell (e.g., - (cd package/doc/sphinx && make html); true) and always return success?

  • Write a maintainer/deploy.sh script (or rake or whatever):

    • It will need to init a git repo in the package/doc/html dir and push this cleanly to the gh-pages branch. (This will also have the advantage that the docs will appear nicely as http://www.mdanalysis.org/mdanalysis/index.html )
    • As commit author we could use the fake Travis CI <TravisCI@mdanalysis.org>.

  • test on a feature branch without screwing up our repo...

Activity

  1. orbeckst
    added
    Priority-Low
    Component-Docs
    testing
    on Aug 7, 2015
  2. orbeckst
    added this to the milestone on Aug 7, 2015
  3. orbeckst
    added a commit that references this issue on Sep 11, 2015

    automatic develop doc deployment

    9817756
  4. orbeckst
    self-assigned this
    on Sep 11, 2015
  5. orbeckst
    added a commit that references this issue on Sep 12, 2015

    automatic develop doc deployment (#386)

    f4bbc95

  6. richardjgowers commented on Sep 12, 2015

    @richardjgowers
    richardjgowers
    on Sep 12, 2015
    Member

    Looks like it didn't work?

    https://travis-ci.org/MDAnalysis/mdanalysis/builds/79970890

    /home/travis/build/MDAnalysis/mdanalysis/package/MDAnalysis/lib/transformations.py:1811: UserWarning: failed to import module _transformations

  warnings.warn("failed to import module " + module_name)

Exception occurred:

  File "/home/travis/build/MDAnalysis/mdanalysis/package/MDAnalysis/lib/distances.py", line 40, in <module>

    from ._distances import (calc_distance_array,

ImportError: No module named _distances

The full traceback has been saved in /tmp/sphinx-err-5NdJCZ.log, if you want to report the issue to the developers.

Please also report this if it was a user error, so that a better error message can be provided next time.

A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!

make: *** [html] Error 1

    It looks like sphinx isn't seeing the Cython modules?

  7. kain88-de commented on Sep 12, 2015

    @kain88-de
    kain88-de
    on Sep 12, 2015
    Member

    They are not installed locally in the package folder. You have to build them with

    Python setup.py build_ext --inplace

    Then the shared object files will be where Sphinx and the Python interpreter can pick them up

  8. orbeckst
    added a commit that references this issue on Sep 12, 2015

    doc deployment: added travis build_ext step

    80bc2a2

  9. orbeckst commented on Sep 12, 2015

    @orbeckst
    orbeckst
    on Sep 12, 2015
    MemberAuthor

    I just pushed 80bc2a2 (directly to develop), which includes @kain88-de 's fix. It means that we're rebuilding the library a second time but that's hopefully not too bad. However, just failed with:

    $ test $TRAVIS_PULL_REQUEST == "false" && test $TRAVIS_BRANCH == "develop" && bash ./maintainer/deploy_docs.sh

bash: ./maintainer/deploy_docs.sh: No such file or directory

    I guess this means that the cd hopping is remembered, i.e. each travis.yml line is not executed in a sub shell. Will fix... but feel free to make this more robust when you look at it. (Sorry for cluttering the history...)

    A bit more annoying is that as soon as we're running multiple CI workers, each one will deploy the docs as well. Maybe there's a way to mark one worker as special or perhaps Travis sets some environment variables that we could use?

  10. richardjgowers commented on Sep 12, 2015

    @richardjgowers
    richardjgowers
    on Sep 12, 2015
    Member

    57eebe1#diff-354f30a63fb0907d4ad57269548329e3R27

    WRT the last point, once we're running multiple jobs, we can nominate one as the "master" one that does the doc push (if TRAVIS_PYTHON == 2.7: push docs)

  11. orbeckst commented on Sep 12, 2015

    @orbeckst
    orbeckst
    on Sep 12, 2015
    MemberAuthor

    For multiple jobs, I found a solution at steveklabnik/automatically_update_github_pages_with_travis_example#5

    I am currently trying out things on the test-docdeploy branch.

  12. orbeckst
    added a commit that references this issue on Sep 12, 2015

    doc deployment: build in place and only run if first job

    3010863
  13. orbeckst
    closed this as completedin c3f3a63on Sep 13, 2015

  14. kain88-de commented on Sep 13, 2015

    @kain88-de
    kain88-de
    on Sep 13, 2015
    Member

    When the module are already build my fix will just cause a copy to the right place and not a full rebuild.

  15. orbeckst
    added 3 commits that reference this issue on Sep 13, 2015

    automatically deploy docs from travis

    d86db3b

    automatically deploy docs from travis

    98a83e1

    Merge pull request #12 from MDAnalysis/ci-deploydocs

    3900181
  16. orbeckst
    added a commit that references this issue on Nov 30, 2015

    removed html docs from repo

    ef62459
  17. lohani2280
    added a commit that references this issue on Feb 17, 2017

    clean up and removal of outdated content: moved proposed New Doc Guid…

    2ff7078
  18. orbeckst
    added 2 commits that reference this issue on May 9, 2017

    changed OAuth token for doc deployment from Travis CI

    3aac336

    changed OAuth token for doc deployment from Travis CI

    3c75391
  19. kain88-de
    added a commit that references this issue on May 9, 2017

    changed OAuth token for doc deployment from Travis CI (#1338)

    36be0d9
  20. orbeckst
    added
    Continuous Integration
    on Jun 17, 2017
  21. orbeckst
    added 2 commits that reference this issue on Oct 12, 2017

    automatic doc deployment

    235351e

    automatic doc deployment

    06254f4
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Component-DocsContinuous IntegrationPriority-Lowtesting

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions