adocman - Automation tool for SourceForge.net DocManager handling

- NAME
- DESCRIPTION
- SYNOPSIS
- OPTIONS

- COMMAND OPTIONS (ACTIONS)
- CONFIGURATION/OUTPUT OPTIONS
- COMMON OPTIONS (USED BY MANY ACTIONS)
- AUTHENTICATION OPTIONS (OPTIONS FOR -login ACTION)
- OPTIONS FOR -list ACTION
- OPTIONS FOR -modify, -download ACTIONS
- OPTIONS FOR -docgroup ACTION

- ERROR LEVELS
- FILES

- CONFIGURATION FILE
- COOKIE FILE (AKA THE COOKIE JAR)
- CONTENT FILES, CONTENT DIRECTORIES

- SECURITY
- AUTHORITATIVE SOURCE
- EXAMPLES
- SUPPORT AND BUGS
- AUTHOR
- PREREQUISITES
- LICENSE
- HISTORY

# NAME

adocman - Automation tool for SourceForge.net DocManager handling

# DESCRIPTION

This program provides a number of functions for manipulating content within the DocManager system of SourceForge.net in an automated fashion (i.e. as part of a project-specific document build system) or in a semi-interactive fashion from the command-line (for sake of saving time).

# SYNOPSIS

adocman [options]

     COMMAND OPTIONS (ACTIONS)
     --login                Login to the SourceForge.net site
     --logout               Logout of the SourceForge.net site
     --help                 Display this help page and exit
                            (use --verbose to provide extended usage info)
     --version              Display version information and exit
                            (use --verbose to see license terms)
     --list=LISTTYPE        Display a list of data from the DocManager system
                            (Types: docgroups, documents, documentsbystate,
                            languages, states)
     --download=TYPE        Retrieve one or more fields of content from the
                            DocManager system and save that data to file
                            (Types: all, state, docgroup, body, summary, title,
                            language, creationdate, updatedate)
     --docgroup=OPERATION   Perform an operation against a DocManager group
                            (Operations: delete, rename, create)
     --modify=TYPE          Modify one or more fields of content within the
                            DocManager system, based on provided content
                            (Types: all, state, docgroup, body, summary, title,
                            language)
     --createnew            Create a new DocManager document
     --prepare              Act as a filter, taking data from STDIN and
                            performing operations to prepare that data for
                            posting to the DocManager; output prepared data
                            to STDOUT.

     CONFIGURATION/OUTPUT OPTIONS
     --verbose              Verbose output
     --quiet                Minimal (reduced) output
     --configfile=FILE      Specify path to configuration file
                            Default: ~/.sfadocmanrc
     --cookiefile=FILE      Specify path to cookie file
                            Default: ~/.sfadocmancookies
     --debug                Ouput debug information and exit
     --hosturl=URL          Specify URL to use in accessing SourceForge.net site
                            Default: https://sourceforge.net
     --hostname=NAME        Specify host name for SourceForge.net site
                            Default: sourceforge.net

     COMMON OPTIONS (USED BY MANY ACTIONS)
     --groupid=GROUPID      Specify the groupid (used for all DocManager opts)
     --docid=DOCID          Specify the docid (used for all doc handling opts)

     AUTHENTICATION OPTIONS (OPTIONS FOR --login)
     --username=USERNAME    Specify the username to use for login
     --password=PASSWORD    Specify the password to use for login
     --interactive          Prompt for the password (interactively)

     OPTIONS FOR --list
     --documentstate=STATE  Specify the document state (by text or by numeric
                            state identifier) for use with
                            --list=documentsbystate

     OPTIONS FOR --modify, --download
     --directoryname=DIR    Specify the directory to be used in obtaining
                            content for modify operation; or the
                            destination directory for download operations.
     --filename=FILE        Specify the file to be used in obtaining content
                            for single-field modify operations.
     --noiddata             Prevent --download from touching doc id data

     OPTIONS FOR --docgroup
     --docgroupid=DOCGROUP  Specify the group id for the DocManager group
                            we wish to manipulate (used for the "delete" and
                            "rename" operations)
     --docgroupname=NAME    Specify the name to be used for the DocManager
                            group (used for the "rename" and "create"
                            operations)

# OPTIONS

## COMMAND OPTIONS (ACTIONS)

--login
  Login to the SourceForge.net site using the provided username and password. Login process saves a cookie to the specified cookiefile. Purges existing cookies from the cookiefile in case we are authenticating as a different user than previously used. Returns an error code of 2 in the event that the login attempt is unsuccessful.

--logout
  Logout of the SourceForge.net site. Ensures that all cookies have been purged from our cookiefile.

-?, --help
  Display usage information for this program. Combine with --verbose to display extended usage information (detailed descriptions of each command-line option). Program exists after displaying usage information.

-V, --version
  Display version information for this program and exit.

--list=LISTTYPE
  Display a list of data derived from the DocManager system. Acceptable values for LISTTYPE include ``docgroups'', ``documents'', ``documentsbystate'', ``languages'' and ``states''. Use of type ``documentsbystate'' requires that the desired state be specified using the --documentstate option.

--download=DOWNLOADTYPE
  Retrieve content from the DocManager system and write it to one or more files. The ``all'' download type is used to perform a download of all content to the directory specified by --directoryname. Other download types, since they deal with only one field, may output their data to either a directory, as specified by the --directoryname option, or to a file, as specified by the --filename option. Document is selected for download based on --groupid and --docid.

  The ``state'' download type is used to download information about the document state (i.e. Active, Pending, Deleted, etc.). The ``docgroup'' download type is used to download information about the document group in which the document currently resides. The ``body'' download type will perform a download of the document body. The ``summary'' and ``title'' download types will perform a download of those fields within the DocManager entry for that document. The ``language'' download type will perform a download of information about the language selected for this document. The ``creationdate'' and ``updatedate'' download types will retrieve a date/time stamp for the document creation and last update, respectively.

  The ``all'' download type encompasses all of the above download types. When performing a download operation to a directory (as specified via --directoryname), files will be named in a manner matching the download type of their content. The ``docid'' file will be overwritten for all operations, for reference purposes (this file is never read by this program), unless the --noiddata option has been used.

--docgroup=OPERATION
  Perform an operation in manipulating the DocManager document groups for the project. Operations include ``delete'', ``rename'', and ``create''. When specifying a document group to remove, you must specify the group by its document group ID, using the --docgroupid option. When renaming a document group, you must specify the group ID (using --docgroupid) and the new name for that group (using --docgroupname). When creating a new document group, you must specify the name for that new group, using the --docgroupname option. --docgroupname=create will return a list of doc group IDs which match the text we supplied for creation (may result in more than one line of output).

--modify=MODIFYTYPE
  Based on provided content, replace one or more fields of content within the document selected by --groupid and --docid. The --all modify type is used to modify all fields for the document to match the contents of the files located in the directory specified using the --directoryname option. Other --modify types, since they deal with only one field, may obtain their data from either a directory, as specified by the --directoryname option, or from a file, as specified by the --filename option. Document is selected for modification based on --groupid and --docid.

  Types for --modify include ``all'', ``state'', ``docgroup'', ``body'', ``summary'', ``title'' and ``language''. If content is being read from a directory (as specified by --directoryname), content will be read from the file whose name matches the modify type name (except ``all'', which reads files for each of the other modify type names).

--createnew
  Create a new DocManager document. Document title, summary, language, body, and docgroup will be set to initial default values. Document language and docgroup will be selected from the list of available options (as returned from the DocManager system). The doc ID for the new document will be returned; you may then proceed to use --modify to alter the document fields from their default values.

--prepare
  Act as a filter, taking data from standard input (STDIN), preparing that data for storage in the DocManager system, then outputting the post-processed data to standard output (STDOUT). Specific operations which will be carried out on the data include the stripping of SourceForge.net site URLs to be absolute local URLs (i.e. to start with a forward slash), purging HTML document header and footer (data before the body start tag, and after the body end tag), and conversion of characters to their proper HTML entities.

## CONFIGURATION/OUTPUT OPTIONS

-v, --verbose
  Display verbose (extended, more detailed) messages, including debugging information and notices as each program operation occurs. Lines which occur specifically as result of the use of --verbose will be preceded by a GMT date/time stamp. If a function is specifically designed to generate a set of output (such as --list functions), that normal output will be preceded by ``-- START OF REQUESTED OUTPUT --'' and will be followed by ``--- END OF REQUESTED OUTPUT ---'', to allow easier parsing of output. Lines which would be generated as result of operation without --verbose will not include a date/time stamp.

-q, --quiet
  Display a reduced set of output, minimized to output specific to the requested operation, and critical/fatal errors which occur during program operation.

--configfile=FILE
  Specify the configfile to be used by the program for configuration settings. Configuration setting names match the names of command-line parameters (without the preceeding dashes). Configuration file will consist of command-line parameters and values (sans dashes), with one line for each parameter/value set.

--cookiefile=FILE
  Specify the cookiefile to be used in saving and accessing the cookie data obtained from the SourceForge.net during the login process.

--debug
  When reporting a bug or issue related to this program, you should perform the same operation as desired (i.e. use the same command-line and the same config file) but add the --debug option to the end of the command-line. This will cause the program to output a dump of recognized configuration details, system information, and script information. This information may be submitted with issue reports as to help ensure an expeditious response.

--hosturl=URL
  Specify the URL to use in accessing SourceForge.net. Defaults to https://sourceforge.net

--hostname=HOSTNAME
  Specify the host name for SourceForge.net. Defaults to sourceforge.net

## COMMON OPTIONS (USED BY MANY ACTIONS)

--groupid=GROUPID
  Specify the groupid whose DocManager whose data you wish to access.

--docid=DOCID
  Specify the docid whose DocManager properties or content you wish to access or modify.

## AUTHENTICATION OPTIONS (OPTIONS FOR --login ACTION)

--username=USERNAME
  Specify the username which should be used in authenticating to the SourceForge.net site. This option is used only during the --login operation.

--password=PASSWORD
  Specify the password which matches the provided username to be used in authenticating to the SourceForge.net site. Password may be specified on the command-line (though a bit unsafe), in the configfile, or you may use the --interactive option to be prompted for the password. This option is used only during the --login operation.

--interactive
  Request that the program prompt you for the password associated with your username. Overrides any password specified in the configfile or on the command-line using the --password option.

## OPTIONS FOR --list ACTION

--documentstate=STATE
  For use in conjuntion with --list=documentsbystate. May be specified using the numeric identifier for the document state, or by the text name for the document state.

## OPTIONS FOR --modify, --download ACTIONS

--directoryname=DIRECTORYPATH
  Specifies the directory whose contents will be read in order to populate the DocManager document fields during a --modify operation. Specifies the directory that will be used as the destination for downloaded content during a --download operation.

--filename=FILENAME
  Specifies the file to be read when obtaining content to be used in a --download or --modify operation that affects only one field.

--noiddata
  Restricts a --download operation from overwriting the DocManager document ID information. By default, the document ID data is overwritten on all download operations which write to a directory (i.e. as specified by the --directoryname option).

## OPTIONS FOR --docgroup ACTION

--docgroupid=DOCGROUP
  Option is used in conjunction with the --docgroup=delete and --docgroup=rename operations, as to select the document group to manipulate.

--docgroupname=GROUPNAME
  Option is used in conjunction with the --docgroup=rename and --docgroup=create operations, as to specify the name you wish to have assigned to the document group.

# ERROR LEVELS

The following error levels are returned upon exit of this program:

     0 success

     1 failure: general (requested DocManager operation failed)

     2 failure: authentication failure

     3 failure: must --login before performing this operation

     4 failure: bad command-line option specified or variable setting problem

     5 failure: error in accessing/creating a file or directory

     6 failure: failed to enter requested input before timeout expired

# FILES

## CONFIGURATION FILE

This program provides the means to specify configuration options through the use of a specially-formatted configuration file. This file is located at ~/.sfadocmanrc by default. This path may be overridden using the --configfile command-line option. This file will contain a list of options, and option and value pairs, with each entry occupying a separate line.

Names for configuration options will match that for command-line parameters. Thus, to enable verbose output (as would be done using the --verbose command-line option), one would add the following line to their configuration file:

      verbose

Options which require value assignments may be specified in the form ``option=value'', with one line for each entry. A sample configuration file follows:

      verbose
      cookiefile=~/.mycookiefile
      groupid=52614
      username=myusername
      password=h128dk19

## COOKIE FILE (AKA THE COOKIE JAR)

Access to the project administration tools on the SourceForge.net site is contingent on successful authentication using a valid username and password combination. Upon successful login, the SourceForge.net provides a cookie containing a session identifier and username information. Subsequent operations on the SourceForge.net site make use of the provided cookie data so as that you need not re-authenticate for each operation.

This program stores cookie data within a dedicated file, specific to this program. The path to this file may be specified using the --cookiefile command-line option (or cookiefile option within your configuration file). This cookie file should be secured, as its contents may be used in accessing the SourceForge.net site as your user account (see the SECURITY section, below). The default path to this cookie file is ~/.sfadocmancookies

## CONTENT FILES, CONTENT DIRECTORIES

This program makes use of user-supplied content for a number of operations, including --modify. A single file is specified using the --filename option. A directory of files (in our download directory format) is specified using the --directoryname option. For all operations which directly read files using the --filename and --directoryname options, this read will occur verbatim; no additional processing is done to the input before its use (so make sure that the file contains the exact desired contents before calling this program).

A number of operations are designed to operate against a ``download directory'' of files. These directories will contain files with the following names, under normal circumstances:

      body - document body

      creationdate - date/time that the DocManager document was created

      docgroup - id and descriptive text for the parent document group

      docid - the DocManager document ID for the document

      language - id and descriptive text for the language property

      state - id and descriptive text for the document state (i.e. Active)

      summary - document summary (description)

      title - document title

      updatedate - date/time that the DocManager document was last updated

# SECURITY

Security is important. Security is particularly important when you are producing materials that will be used by many people, such as Open Source software and documentation. This section of the documentation for adocman provides a basic overview of the security-related issues in using this application, as well as a general risk analysis. All users are encouraged to consider this section carefully before using adocman, and certainly before implementing a solution based on adocman.

When obtaining software over the Internet, it is important to take precautions to ensure that you have obtained an original, verbatim version of that software, which has not been modified or tainted by third parties. We encourage you to verify the MD5 sum of this application with those posted on the SourceForge.net Site Documentation project web site at: http://sitedocs.sourceforge.net

If you choose to build a custom version of this software, please contact the Site Documentation team, so as that we will have your name and e-mail address on-file in case issues arise. Further, please provide the means for your end-users to verify the integrity of your releases.

This software takes very few precautions to protect your data, compared to those steps which a paranoid might take. The security burden you face in using this application lays largely with you. Consider this carefully before using this application; only you can prevent forest fires.

By default, data will be sent to the SourceForge.net site using SSL-protected HTTP. This SSL support is based on perl's Crypt:SSLeay, a module which provides cryptographic functions. Use of cryptography is legal in the United States, to the best of our knowledge, but may be illegal in some parts of the world. Verify that your actions meet the requirements of your local, regional, and national laws.

SourceForge.net user accounts are provided extensive power over portions of a project, once they have been added to the developer list. This program will essentially be used by either interactive users (to save time over the web interface when performing repetitive operations), or in non-interactive scripts (such as part of a documentation build system which automatically posts to the SourceForge.net DocManager for that project).

We recommend that this program only be used from secure workstations, and that this program NOT be used on shared hosts (such as the SourceForge.net project shell servers). If you do intend to make use of this program in automated scripts, we recommend that you create a separate user account for your project (projectname_docman or somesuch) to use in these automated operations. Such a user would be granted only the DocManager Editor permissions. Further steps should be taken to protect your CVS repository (all developers are automatically granted write permissions to the repository), through the use of the cvs_acls script, provided in the contrib directory of the CVS software distribution. Taking these steps will significantly reduce the security impact of compromise of the account, essentially limiting the impact to the DocManager system itself.

Backups of your DocManager content, and all other project data, should be made regularly, as to protect the long-term integrity of your data and the extensive time investment your project team has made. You may wish to use this program in automating backups of your DocManager content and properties; if you choose to do so, please review your backups on a regular basis to ensure that the archived data is as expected (this program is not infallable; it may have data corrupting bugs which have not yet been reported or fixed).

This software may contain bugs. This software depends on a number of third-party modules available from the CPAN module repository. Those third-party modules may contain bugs of their own. Everyone in the Open Source community is working to improve the quality of the software available to end-users. As an end-user of this software, we ask two things of you. First, please ensure that you regularly update your copy of this software; future releases will contain new enhancements and bugfixes which may be important to your use of this program. Second, please help us to improve the quality of this program by reporting any bugs that you encounter (see the SUPPORT AND BUGS section of this document, below).

We recommend that you not use this software in a matter which a SourceForge-based service provider might feel is abusive. Such behavior would reflect badly upon you, and may be considered a breach of the Terms of Service for that provider. Test your scripts carefully to ensure that they behave as expected. Review the automated output of your scripts and cron jobs to ensure they are functioning as expected.

This program changes its name ($0) in the process listing (for platforms which support such a change), but it may still be possible for another user to derive information about the command-line arguments fed to this program, when it is used on a multi-user system. If you do choose to operate this program in a non-dedicated environment, consider storing your password data in a configuration file, rather than specifying it on the command line (and be sure to set the permissions on the configuration file to protect it from prying eyes).

We recommend that you set a dedicated, unique password for your SourceForge.net user account(s), and that you change this password periodically. Passwords should not be shared between SourceForge.net and other sites on which you hold accounts.

Finally, please ensure that the permissions are properly set on any configuration files, cookie files and data files for this program. Using this program, it is possible to extract information from the DocManager system which is visible only to DocManager Editors and Project Administrators for your project; you may wish to set the permissions on those files to ensure that other users are not able to read those documents. Cookie files and configuration files should be well protected; their contents may be used to authenticate to the SourceForge.net site and perform operations as you.

Evaluating these concerns is an important step to implementing this software for your project. Should you have specific concerns, feel free to research them further, then contact us (see the ``SUPPORT AND BUGS'' section, below).

# AUTHORITATIVE SOURCE

The original version of this program may be found in the materials provided from the SourceForge.net Site Documentation project (sitedocs) on the SourceForge.net site. The latest version of this program may be found in the CVS repository for the sitedocs project on SourceForge.net. The sitedocs project pages may be accessed at: http://sourceforge.net/projects/sitedocs

# EXAMPLES

The following are examples for using this program to manage the content within the DocManager for your project. It is presumed that you have a valid SourceForge.net user account, which is a listed developer or project administrator on the project in question. Developers who will be using this tool must be provided with DocManager 'Editor' status under the 'User Permissions' section of the Admin interface for the project. The group ID for the project may be derived from the URL for the Admin page for the project, or by viewing the Project Admin page for the project (look for the text ``Your Group ID is: xxxxxx'').

To login to the SourceForge.net site via the command-line:

      adocman --username=myusername --password=mypassword --login --groupid=8675309

To login to the SourceForge.net site, and be prompted to enter your password interactively:

      adocman --username=myusername --interactive --login --groupid=8675309

To obtain a list of documents available to your end-users:

      adocman --groupid=8675309 --list=documentsbystate --documentstate=Active

To obtain the same list, but only the doc IDs for each doucment:

      adocman --groupid=8675309 --list=documentsbystate --documentstate=Active -q

To download a complete set of information about a document (document ID 5551212) from our DocManager (group ID 8675309) to a local directory (~/myworkdirectory):

      adocman --directoryname=~/myworkdirectory --docid=5551212 \
        --groupid=8675309 --download=all

To download just the body of a document to a local file (~/myoutputfile):

      adocman --filename=~/myoutputfile --docid=5551212 --groupid=8675309 \
        --download=body

To modify the body of a document so it matches our local copy, which is stored in a directory we previously had downloaded to (using --directoryname):

      adocman --directoryname=~/myworkdirectory --docid=5551212 \
        --groupid=8675309 --modify=body

To prepare an existing file for posting to the DocManager:

      adocman --prepare < original.html > prepared.html

To logout of SourceForge.net:

      adocman --logout

To obtain output for debugging a problem, perform the same command as originally tested, but first add the --verbose flag, and determine whether you are able to solve the issue on your own. If the problem persists, see the ``SUPPORT AND BUGS'' section, below.

# SUPPORT AND BUGS

This program was written by a member of the SourceForge.net staff team. This software has been released under an Open Source license, for the greater benefit of the SourceForge.net developer community.

The SourceForge.net Site Documentation project is the caretaker of this software. Issues related to the use of this program, or bugs found in using this program, may be reported to the SourceForge.net Site Documentation project using their Support Request Tracker at: https://sourceforge.net/tracker/?func=add&group_id=52614&atid=467457

Any support that is provided for this program is provided as to further enhance the stability and functionality of this program for SourceForge.net users. The SourceForge.net Site Documentation project makes use of this software for its own internal purposes, in managing the Site Documentation collection for the SourceForge.net site.

# AUTHOR

Jacob Moorman <moorman@sourceforge.net> Nathan Oostendorp <oostendo@sourceforge.net>

# PREREQUISITES

`LWP::UserAgent`, `HTML::TokeParser`, `Crypt::SSLeay`, `Digest::MD5`, `Term::ReadKey`

These prerequisites may be installed in an interactive, but automated fashion through the use of perl's CPAN module, invoked as:

      perl -MCPAN -e shell;

# LICENSE

Copyright (c) 2002 Open Source Development Network, Inc. (``OSDN'')

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the ``Software''), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

1. The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

2. Neither the names of VA Software Corporation, OSDN, SourceForge.net, the SourceForge.net Site Documentation project, nor the names of its contributors may be used to endorse or promote products derived from the Software without specific prior written permission of OSDN.

3. The name and trademarks of copyright holders may NOT be used in advertising or publicity pertaining to the Software without specific, written prior permission. Title to copyright in the Software and any associated documentation will at all times remain with copyright holders.

4. If any files are modified, you must cause the modified files to carry prominent notices stating that you changed the files and the date of any change. We recommend that you provide URLs to the location from which the code is derived.

5. Altered versions of the Software must be plainly marked as such, and must not be misrepresented as being the original Software.

6. The origin of the Software must not be misrepresented; you must not claim that you wrote the original Software. If you use the Software in a product, an acknowledgment in the product documentation would be appreciated but is not required.

7. The data files supplied as input to, or produced as output from, the programs of the Software do not automatically fall under the copyright of the Software, but belong to whomever generated them, and may be sold commercially, and may be aggregated with the Software.

8. THE SOFTWARE IS PROVIDED ``AS IS'', WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE OR DOCUMENTATION.

This Software consists of contributions made by OSDN and many individuals on behalf of OSDN. Specific attributions are listed in the accompanying credits file.

# HISTORY

2002-05-25 Started initial design, under name autodocpost

2002-05-30 Completed initial proof-of-concept (post/create documents only)

2002-05-30 Completed design document for full-featured implementation

2002-05-31 Started implementation of full-featured design under name adocman

2002-06-01 Completed version 0.01 - bulk of core functionality

2002-06-02 Completed version 0.02 - added --modify, additional --list's

2002-06-02 Completed version 0.03 - added --createnew, additional --list's

2002-06-02 Completed version 0.04 - POD clean-up, diversified error levels

2002-06-03 Completed version 0.05 - hardcoded URLs removed, added --prepare

2002-06-03 Completed version 0.06 - added --interactive, --debug

2002-06-04 Completed version 0.07 - POD, add --host, def. config handling

2002-06-04 Completed version 0.08 - general clean-up, POD

2002-12-03 Interim version 0.09 - complete back-out of interim changes

2002-12-03 Completed version 0.10 - move to classes, added xml export tool