
      fetch.txt                                          update 25 Jan 2002

      NAME
            fetch - retrieves database entries by name or accession number
      
      SYNOPSIS
            fetch                                    {interactive mode}
            fetch [options] namefile [output file]   {batch mode}
      
      DESCRIPTION
	    fetch retrieves one or more entries from a database.

      Interactive mode: fetch prompts the user to set search parameters, 
      using an interactive menu:
      ___________________________________________________________________
                           FETCH - Version   7 Feb 94                    
         Please cite: Fristensky (1993) Nucl. Acids Res. 21:5997-6003
      ___________________________________________________________________
      Namefile:
      Outfile: 
      Database:
      -------------------------------------------------------------------
         Parameter              Description                      Value

      1) Name/Acc    Name or Accession sequence to get             
      2) Namefile    Get list of sequences from Namefile
      3) WhatToGet   a:annotation s:sequence b:both                b
      4) Database    g:GenBank p:PIR                               g
                     G:GenBank dataset   P:PIR dataset
      5) Outfile     Send all output to a single file (Outfile)
      6) Files       f:Send each entry to a separate file          f
         -------------------------------------------------------------
         Type number of your choice or 0 to continue:

            After all parameters have been set, type 0 to commence the search.
            Messages regarding the progress of the search will be printed.

            (1,2) Which entries to get?
            If you want to get a single entry, option 1 lets you type in the 
            name of that entry, without having to create a namefile. To get
            more than one entry, choose option 2, and specify the name of a
            file containing sequence names or accession numbers. 

            namefile is a file containing one or more sequence names or
	       accession numbers, each on a separate line. Names and accession
            numbers can even be interspersed, in upper or lowercase, and in
            any order. For example, the namefile prp.nam might contain

            ; plant pathogenesis related proteins
            ;   (these are sample comment lines)
            ;   note that any line containing a semicolon is ignored
            x06362
            x05454
            TOBPR1A1
            ;  comments can be interspersed with names.
            PUMPR13
            tobpr1ar
             
            Options 1 & 2 are mutually exclusive. Setting one will negate the
            other. If option 2 is chosen, the name of the namefile will appear
            at the top of the menu. 

            (3) WhatToGet
            Use this option to specify whether to get annotation, sequence, 
            or both (default=both).

            (4) Database
            Use this option to select the database. (default=GenBank).
            G and P select user-created database subsets containing GenBank
            or PIR entries, respectively. It is assumed that the database
            has been split into .ano, .wrp and .ind files using splitdb.
            For example, if you had created a database subset called PR1.pir,
            splitdb would create PR1.ano, PR1.wrp and PR1.ind. These are
            the files actually read by FETCH. When prompted for the name
            of the database, simply type "PR1", without a file extension.
            (If you do type a file extension, it will be ignored).

            (5, 6) Where to send output
            By default, option 6 is set to f, and each entry will be written to
            a separate file, where the name of the file is the name of the
            entry, followed by a file extension. If a complete entry is
            retrieved, the file extension will indicate the type of database
            (GenBank: .gen;  PIR: .pir, Vecbase: .vec; LiMB: .LiMB). If only
            annotation or sequence are retrieved, the file extensions will be
            .ano or .wrp, respectively. Using the default, the namefile above
            would create the following files:
  
            PUMPR13.gen
            TOBPR1A1.gen
            TOBPR1AR.gen
            TOBPR1CR.gen
            TOBPR1PS.gen

            By choosing option 5, you can specify the name of an output file
            for all entries to go to. This filename will appear at the top
            of the menu. Obviously, options 5 & 6 are mutually exclusive.
            Note entries retrieved are writen in alphabetical order (sorting by 
            ASCII values), not the order in which they appeared in namefile.

            (Note for remote users only: -f will only work for a single
            name/accession supplied in 1). -f IS NOT ENABLED FOR NAMEFILES 
            specified in 2).)
           
       Batch mode:
	    Although it is transparent to the user, all fetch really does
	    is call getloc, saving the user the trouble of knowing which
	    database files to retrieve sequences from, or of having to
	    execute getloc multiple times to retrieve sequences from
	    different database files. Thus, the options are identical to those
            for getloc:

            -a    Write annotation portions of entries only, terminated by '//'.
            -s    Write sequence data only, in Pearson (.wrp) format.
            -f    Write each entry to a separate file.
            -g    GenBank (default)
            -e    EMBL  {not implemented}
            -p    PIR (NBRF)
            -G    GenBank_dataset
            -P    PIR_dataset

            If -f is not specified, outfile must be specified.

            -L    force execution of findkey on local host even if 
                  $XYLEM_RHOST is set. See "REMOTE EXECUTION" below


         PIR_dataset
         GenBank_dataset
         This can be either a file of PIR entries, a file of GenBank entries,
         or a XYLEM dataset created by splitdb. A file of PIR entries must
         have the file extension ".pir". A file of GenBank entries must have
         the file extension ".gen". A XYLEM dataset contains PIR entries split
         among three files by splitdb: annotation (.ano), sequence (.wrp)
         and index (.ind). These file extensions must be used!

         When specifying a split dataset, only the base name needs to be
         used. For example given a XYLEM dataset consisting of the files
         myset.ano, myset.wrp and myset.ind, the following two commands
         are equivalent:

         fetch -P myset  something.nam something.pir
         fetch -P myset.ano something.nam something.pir

         If the original .pir file had been used, the command would have
         been

         fetch -P myset.pir something.nam something.pir

         The ability to work directly with .gen or .pir files is quite
         convenient. However, since FETCH needs to work with a split
         FETCH automatically splits .pir or .gen files into .ano, .wrp
         and .ind files, which are removed when finished. This requires
         extra disk space and execution time, which could be significant
         for large datasets.
         
     EXAMPLES
	Batch example:
	fetch  -f chitinase.nam
            will retrieve annotation and sequence for sequences listed in 
            chitinase.nam from GenBank, writing each entry to a separate file
            with the extension .gen.

        fetch -s -p test.nam test.wrp
            will retrieve sequence data only for the entries listed in test.nam,
            from PIR, and write all sequences to a Pearson format file
            (ie. readable by fasta) with the name test.wrp.

        fetch -G sample sample.nam new.gen
        fetch -G sample.ano  sample.nam new.gen
            Assumes that a set of GenBank entries has been split by splitdb
            into sample.ano sample.wrp and sample.ind. The entries listed in
            sample.nam are written to new.gen.


      FILES
        Database files:
          The directories for database files are specified by the environment
          variables $GB (GenBank) and $PIR (PIR/NBRF) .

          Index files are $GB/gbacc.idx for GenBank (this file is supplied
          with each GenBank release), while the other databases
          use .ind files generated by splitdb. Split database files MUST
          have the following file extensions: .ano {annotation}, .wrp
          {sequence} and .ind {index}. Thus, when creating database files
          for pir1.dat with splitdb, the output files should be pir1.ano,
          pir1.wrp and pir1.ind.

        Temporary files:
          NAMEFILE.fetch
          PRELIMINARY.fetch
          TMP.fetch
          FOUND.fetch
          FETCHDIR  {temporary directory}

     REMOTE EXECUTION
          Where the databases can not be stored locally, FETCH can call
          FETCH on another system and retrieve the results. To run
          FETCH remotely, your .cshrc file should contain the following
          lines:

          setenv XYLEM_RHOST remotehostname
          setenv XYLEM_USERID remoteuserid

          where remotehostname is the name of the host on which the 
          databases reside (in XYLEM split format) and remoteuserid
          is your userid on the remote system. When run remotely, 
          your local copy of FETCH will generate the following 
          commands:

          rcp filename $XYLEM_USERID@$XYLEM_HOST:filename
          rsh $XYLEM_RHOST -l $XYLEM_USERID fetch ...
          rcp $XYLEM_USERID@$XYLEM_HOST:outputfilename outputfilename
          rsh $XYLEM_RHOST -l $XYLEM_USERID $RM temporary_files

          Because FETCH uses rsh and rcp, your home directory on both
          the local and remote systems must have a world-readable
          file called .rhosts, containing the names of trusted remote
          hosts and your userid on each host. Before trying to get
          FETCH to work remotely, make sure that you can rcp and 
          rsh to the remote host.

          Obviously, remote execution of FETCH implies that FETCH
          must already be installed on the remote host. When FETCH
          runs another copy of FETCH remotely, it uses the -L option
          (findkey -L) to insure that the remote FETCH job executes,
          rather than calling yet another FETCH on another host.


          ---------- Remote execution on more than 1 host -----------
          If more than 1 remote host is available for running FINDKEY
          (say, in a clustered environment where many servers mount
          a common filesystem) the choice of a host can be determined
          by the csh script choosehost, such that execution of 
          choosehost returns the name of a remote server. To use this
          approach, the following script, called 'choosehost' should
          be in your bin directory:

          #!/bin/csh              
          # choosehost - choose a host to use for a remote job. 
          # This script rotates among servers listed in .rexhosts, 
          # by choosing the host at the top of the list and moving
          # it to the bottom.

          #Rotate the list, putting the current host to the bottom.
          set HOST = `head -1 $home/.rexhosts`
          set JOBID = $$
          tail +2 $home/.rexhosts > /tmp/.rexhosts.$JOBID
          echo $HOST >> /tmp/.rexhosts.$JOBID
          /usr/bin/mv /tmp/.rexhosts.$JOBID $home/.rexhosts

          # Write out the current host name
          echo $HOST

          You must also have a file in your home directory called
          .rexhosts, listing remote hosts, such as

          graucho.cc.umanitoba.ca
          harpo.cc.umanitoba.ca
          chico.cc.umanitoba.ca
          zeppo.cc.umanitoba.ca

          Each time choosehost is called, choosehost will rotate the
          names in the file. For example, starting with the .rexhosts
          as shown, it will move graucho.cc.umanitoba.ca to the bottom
          of the file, and write the line 'graucho.cc.umanitoba.ca'
          to the standard output. The next time choosehosts is
          run, it would write 'harpo.cc.umanitoba.ca', and so on.

          Depending on your local configuration, you may wish to 
          rewrite choosehosts. All that is really necessary is that
          echo `choosehost` should return the name of a valid host.

          Once you have installed choosehost and tested it, you can
          get FINDKEY to use choosehost simply by setting
          
          setenv XYLEM_RHOST choosehost

          in your .cshrc file.

          ---------------  Remote filesystems -----------------------
          Finally, an alternative to remote execution is to remotely mount
          the file system containing the databases across the network.
          This has the advantage of simplicity, and means that the 
          databases are available for ALL programs on your local
          workstation. However, it may still be advantageous to run
          FETCH remotely, since that will shift much of the computational
          load to another host.

    BUGS
          When retrieving entries directly from GenBank, FETCH uses the
          Accession Number index file gbacc.idx. In this case, FETCH
          can retrieve all entries containing a given accession number.
          This capability makes it possible to retrieve an entry using a
          secondary accession number. However if more than one entry
          share a secondary accession number, all of those entries will
          be retrieved. While this behavior might be a bit of an 
          annoyance at times, it can also be useful because it alerts
          the user to the presence of other, related entries that might
          be of interest.

    SEE ALSO
            getloc features

     AUTHOR
       Dr. Brian Fristensky
       Dept. of Plant Science
       University of Manitoba
       Winnipeg, MB  Canada  R3T 2N2
       Phone: 204-474-6085
       FAX: 204-474-7528
       frist@cc.umanitoba.ca

     REFERENCE
       Fristensky, B. (1993) Feature expressions: creating and manipulating
       sequence datasets. Nucleic Acids Research 21:5997-6003.
