help:faq [2009/12/04 11:09]
hpm (172.22.10.43) Update "Perl: I installed SOAP::Lite, but I get an error?" to describe issues with SOAP::Lite 0.710.10. |
help:faq [2013/08/21 11:06] (current)
hpm (172.22.68.212) |
| - | ====== FAQ ====== | + | ====== Frequently Asked Questions (FAQ) ====== |
| | | | |
| - | Answers to frequently asked questions (FAQs) about EBI's Web Services. | + | Answers to frequently asked questions (FAQs) about the Web Services provided by EMBL-EBI. |
| | | | |
| - | If you have a question that is not answered here please contact [[http://www.ebi.ac.uk/support/index.php?query=WebServices|EBI Support]]. | + | If you have a question that is not answered here please contact [[http://www.ebi.ac.uk/support/index.php?query=WebServices|EMBL-EBI Support]]. |
| | + | |
| | + | ===== Staying Informed ===== |
| | + | |
| | + | To stay informed about updates and changes to the EMBL-EBI Web Services, and scheduled maintenance to the EMBL-EBI website and services we suggest: |
| | + | |
| | + | * Keep an eye on the [[:about:news|Web Services News]] ([[http://www.ebi.ac.uk/Tools/webservices/feed.php?linkto=current&mode=list&ns=about&type=rss2|RSS]] or [[http://www.ebi.ac.uk/Tools/webservices/feed.php?linkto=current&mode=list&ns=about&type=atom1|Atom]]). |
| | + | * Subscribe to the [[http://listserver.ebi.ac.uk/mailman/listinfo/ws-users|ws-users mailing list]]. This is a low volume mailing list containing service announcements and notification of appropriate training events. |
| | + | * Follow [[https://twitter.com/#!/emblebi|emblebi]] and/or [[https://twitter.com/#!/emblebies|emblebies]] on [[http://twitter.com/|Twitter]]. |
| | + | * "Like" [[http://www.facebook.com/EMBLEBI|EMBL-EBI]] on [[http://www.facebook.com/|Facebook]]. |
| | | | |
| | ===== How do I use these services? ===== | | ===== How do I use these services? ===== |
| | === Client Development === | | === Client Development === |
| | | | |
| - | If you want to develop you own clients to the EBI Web Services, then you will need to set-up an appropriate development environment for your chosen programming language and select an appropriate SOAP library. | + | If you want to develop you own clients to the [[:|EMBL-EBI Web Services]], then you will need to set-up an appropriate development environment for your chosen programming language and select an appropriate REST or SOAP took-kit. |
| | | | |
| - | A number of [[tutorials:intro|tutorials]] are provided covering most common programming languages: | + | A number of [[:tutorials:06_Programming|tutorials]] are provided covering most common programming languages and popular tool-kits: |
| - | * [[tutorials:csharp|C#]] | + | * [[:tutorials:06_Programming:C_CPP|C/C++]] |
| - | * [[tutorials:java|Java]] | + | * [[:tutorials:06_Programming:dot_net:Csharp|C#]] |
| - | * [[tutorials:perl|Perl]] | + | * [[:tutorials:06_Programming:Java]] |
| - | * [[tutorials:php|PHP]] | + | * [[:tutorials:06_Programming:Perl]] |
| - | * [[tutorials:python|Python]] | + | * [[:tutorials:06_Programming:PHP]] |
| - | * [[tutorials:ruby|Ruby]] | + | * [[:tutorials:06_Programming:Python]] |
| - | * [[tutorials:vb.net|Visual Basic .NET]] | + | * [[:tutorials:06_Programming:Ruby]] |
| | + | * [[:tutorials:06_Programming:dot_net:VB.NET|Visual Basic .NET]] |
| | | | |
| | === Existing Clients === | | === Existing Clients === |
| | | | |
| - | If you want to download a client and use the service right away then see the clients pages for the service of interest ([[about:webservices|list]]). The clients page (e.g. [[clients:interproscan|InterProScan Clients]]) provides a list of clients and details of the pre-requisites. | + | If you want to download a client and use the service right away then see the service documentation pages for the service of interest ([[:|list]]). The service page (e.g. [[:services:pfa:iprscan_soap|InterProScan (SOAP)]]) provides a list of clients and details of the pre-requisites. |
| | | | |
| - | For most services we provide command-line clients for Java and Perl. For a few selected services a number of other clients are also available (e.g. [[clients:dbfetch|WSDbfetch]]). Which client you want to use depends on the platform. Currently we recommend using: | + | For most services we provide command-line clients for .NET, Java and Perl. For a few selected services a number of other clients are also available, for example: |
| | + | * [[:services:EB-eye]] |
| | + | * [[:services:pfa:iprscan_rest|InterProScan (REST)]] |
| | + | * [[:services:pfa:iprscan_soap|InterProScan (SOAP)]] |
| | + | * [[:services:sss:NCBI BLAST (REST)]] |
| | + | * [[:services:sss:NCBI BLAST (SOAP)]] |
| | + | * [[:services:dbfetch_rest|WSDbfetch (REST)]] |
| | + | * [[:services:dbfetch|WSDbfetch (SOAP)]] |
| | + | Which client you want to use depends on the platform. Currently we recommend using: |
| | | | |
| | * MS Windows | | * MS Windows |
| | ===== What tools do I need? ===== | | ===== What tools do I need? ===== |
| | | | |
| - | If you want to download a client and get running with a service quickly, then see the clients page for the service of interest ([[about:webservices|list]]) for details of any existing clients. | + | If you want to download a client and get running with a service quickly, then see the clients page for the service of interest ([[:|list]]) for details of any existing clients. |
| | | | |
| | If you want to develop you own clients, or services, you will need to choose a programming language and, if SOAP is not included with the language, a SOAP library. There are a wide selection of possibilities, some of the more common solutions are: | | If you want to develop you own clients, or services, you will need to choose a programming language and, if SOAP is not included with the language, a SOAP library. There are a wide selection of possibilities, some of the more common solutions are: |
| | * Ruby with [[http://raa.ruby-lang.org/project/soap4r/|soap4r]] | | * Ruby with [[http://raa.ruby-lang.org/project/soap4r/|soap4r]] |
| | | | |
| - | Tutorials on using a selection of these languages and libraries can be found in the [[http://www.ebi.ac.uk/Tools/webservices/tutorials/intro|tutorials pages]]. | + | Tutorials on using a selection of these languages and libraries can be found in the [[:tutorials:06_Programming|tutorials pages]]. |
| | | | |
| | ===== How can I analyse multiple sequences? ===== | | ===== How can I analyse multiple sequences? ===== |
| | | | |
| - | The Web Services provided by EBI are limited to one sequence, or one set of sequences, per job. Thus if you wish to analyse multiple sequences you will have to submit multiple jobs. The simplest way to do this is to create a set of files containing one sequence per file, and then loop over the files submitting each one, and getting the result. | + | The Web Services provided by EMBL-EBI are limited to one sequence (e.g. the query sequence for a sequence similarity search), or one set of sequences (e.g. the set of sequences to be aligned for a multiple sequence alignment), per job. Thus if you wish to analyse multiple sequences you will have to submit multiple jobs. |
| | | | |
| - | To split a file containing multiple fasta formatted sequences use the following scripts: | + | **Please note**: We kindly ask all users to submit **NO MORE THAN 25 JOBS AT THE TIME AND NOT TO SUBMIT MORE JOBS UNTIL YOU HAVE OBTAINED RESULTS FOR THE LAST 25**. There are many people using these services and a fair share policy has been implemented that allows us to block users that submit jobs in a manner that prevents others from using the service. This block may affect access to the EMBL-EBI Web Services for an entire organisation or a class B or C subnet. Also make sure you **USE A REAL EMAIL ADDRESS** in your submissions. Using a fake email means we cannot contact you and will very likely result in your jobs being killed and your IP, Organisation or entire domain being black-listed. We do apologise for any inconvenience this may cause. |
| - | * UNIX and UNIX-like environments: [[http://www.ebi.ac.uk/Tools/webservices/downloads/util/fsplit.sh|fsplit.sh]] | + | |
| - | * MS Windows: [[http://www.ebi.ac.uk/Tools/webservices/downloads/util/fsplit.bat|fsplit.bat]] | + | |
| | | | |
| - | These scripts require the ''csplit'' program. The ''csplit'' program is available as part of the [[http://www.gnu.org/software/coreutils/|GNU coreutils]] package, for MS Windows a pre-complied version is available: [[http://www.ebi.ac.uk/Tools/webservices/downloads/util/csplit.zip|csplit.zip]]. | + | === Multiple sequences per-file === |
| | + | |
| | + | For some services the supplied clients support input of multiple sequences in the [[http://www.ebi.ac.uk/2can/tutorials/formats.html#fasta|fasta sequence format]]. This has to be specifically enabled using a flag, see the client's help/usage information (typically accessed using the ''--help'' option) for details. |
| | + | |
| | + | === File per-sequence === |
| | + | |
| | + | A simple way to do this is to create a set of files containing one sequence per file, and then loop over the files submitting each one, and getting the result. |
| | + | |
| | + | To split a file containing multiple fasta formatted sequences into a set of files each containing a single sequence the following methods can be used: |
| | + | * Use the [[http://emboss.open-bio.org/rel/rel6/apps/seqretsplit.html|EMBOSS seqretsplit]] program, part of [[http://emboss.open-bio.org/|EMBOSS]], a collection of sequence manipulation and analysis tools. |
| | + | * Use a script which uses the ''csplit'' utility. The ''csplit'' program is available as part of the [[http://www.gnu.org/software/coreutils/|GNU coreutils]] package, for MS Windows a pre-complied version is available: [[http://www.ebi.ac.uk/Tools/webservices/downloads/util/csplit.zip|csplit.zip]]. See the following scripts for example usage: |
| | + | * UNIX and UNIX-like environments: [[http://www.ebi.ac.uk/Tools/webservices/downloads/util/fsplit.sh|fsplit.sh]] |
| | + | * MS Windows: [[http://www.ebi.ac.uk/Tools/webservices/downloads/util/fsplit.bat|fsplit.bat]] |
| | | | |
| | To following scripts illustrate how to submit the jobs: | | To following scripts illustrate how to submit the jobs: |
| - | * UNIX and UNIX-like environments: [[http://www.ebi.ac.uk/Tools/webservices/downloads/util/wsbatch.sh|wsbatch.sh]] | + | * UNIX and UNIX-like environments: |
| | + | * Dispatcher: [[http://www.ebi.ac.uk/Tools/webservices/downloads/util/wsbatch.sh|wsbatch.sh]] |
| | + | * JDispatcher: [[http://www.ebi.ac.uk/Tools/webservices/download_clients/util/jdispatcher_file_per-sequence_batch.sh|jdispatcher_file_per-sequence_batch.sh]] |
| | + | |
| | + | The following parallel job running tools may also be useful: |
| | + | |
| | + | * [[http://www.gnu.org/software/make/|GNU make]] |
| | + | * [[http://www.gnu.org/software/parallel/|GNU Parallel]] |
| | + | * [[http://www.maier-komor.de/xjobs.html|xjobs]] |
| | + | |
| | + | === Workflow systems === |
| | + | |
| | + | Workflow systems (e.g. [[https://kepler-project.org/|Kepler]], [[http://www.taverna.org.uk/|Taverna]] or [[http://www.trianacode.org/|Triana]]) support a number of forms of automated bulk analysis. See [[http://www.ebi.ac.uk/Tools/webservices/tutorials/07_workflows|Workflows and Web Services]] for further details. |
| | + | |
| | + | === Dedicated Tools === |
| | + | |
| | + | Dedicated analysis tools, such as [[http://www.blast2go.com/|Blast2GO]], may also use web services to provide some functionality or to perform a specific analysis. See their documentation for details. |
| | | | |
| | ===== How do I specify a proxy? ===== | | ===== How do I specify a proxy? ===== |
| | </code> | | </code> |
| | | | |
| - | For SOAP::Lite you can also set the HTTP proxy directly, see the [[tutorials:perl|Perl tutorial]] for details. | + | For SOAP::Lite you can also set the HTTP proxy directly, see the [[:tutorials:06_Programming:Perl|Perl tutorial]] for details. |
| | | | |
| | If you cannot set the environment variables directly then you can set them from your Perl script. For example: | | If you cannot set the environment variables directly then you can set them from your Perl script. For example: |
| | ===== Why do you need my e-mail address? ===== | | ===== Why do you need my e-mail address? ===== |
| | | | |
| - | We require a valid e-mail address so we can contact you in the event of problems with the service which affect your jobs. As described in our [[http://www.ebi.ac.uk/Information/termsofuse.html|Terms of Use]] this information is not provided to anyone else, nor will we contact you except for the reasons stated above. | + | We require a valid e-mail address so we can contact you in the event of: |
| | + | * Problems with the service which affect your jobs. |
| | + | * Scheduled maintenance which affects services you are using. |
| | + | * Deprecation and retirement of a service you are using. |
| | + | As described in our [[http://www.ebi.ac.uk/about/terms-of-use|Terms of Use]] this information is not provided to anyone else, nor will we contact you except for the reasons stated above. |
| | | | |
| - | If you use a fake e-mail address then we will not be able to contact you and will very likely result in your jobs being killed and your IP, Organisation or entire domain being black-listed. | + | If you use a fake e-mail address then we will not be able to contact you and will very likely result in your jobs being killed and your IP, Organisation or entire domain being black-listed, see [[http://www.ebi.ac.uk/about/terms-of-use|Terms of Use]]. |
| | | | |
| - | ===== Staying Informed ===== | |
| - | | |
| - | To stay informed about changes to the EBI web services and scheduled maintenance we recommend: | |
| - | | |
| - | - Keep an eye on the [[about:news|News page]]. There are RSS news feeds associated with the page if you want to see changes in your news reader. | |
| - | - The [[http://listserver.ebi.ac.uk/mailman/listinfo/ws-users|ws-users mailing list]]. This is a low volume mailing list containing service announcements and notification of appropriate training events. | |
| | | | |
| | ===== Perl: Installing modules on MS Windows? ===== | | ===== Perl: Installing modules on MS Windows? ===== |
| | ===== Perl: I installed SOAP::Lite, but I get an error? ===== | | ===== Perl: I installed SOAP::Lite, but I get an error? ===== |
| | | | |
| - | The latest version of [[http://search.cpan.org/dist/SOAP-Lite/|SOAP::Lite]] (0.710.10) contains changes that affect compatibility with our services. Typical error messages indicating compatibility problems include: | + | Various versions of [[http://search.cpan.org/dist/SOAP-Lite/|SOAP::Lite]] contains bugs which cause compatibility issues with our services. Typical error messages indicating compatibility problems include: |
| | * <code>Can't find method element in the message at | | * <code>Can't find method element in the message at |
| - | /ebi/extserv/bin/perl-5.8.3/lib/site_perl/5.8.3/SOAP/Lite.pm line 2311.</code> This error is caused by using code developed for SOAP::Lite 0.60a (or eariler) with a later version of the library. As noted in the [[..:tutorials:perl|Perl tutorial]], some services require that the parameters are passed in a slightly different way. | + | /ebi/extserv/bin/perl-5.8.3/lib/site_perl/5.8.3/SOAP/Lite.pm line 2311.</code> This error is caused by using code developed for SOAP::Lite 0.60a (or eariler) with a later version of the library. As noted in the [[:tutorials:06_Programming:perl|Perl tutorial]], some services require that the parameters are passed in a slightly different way. |
| | + | * <code>Type 'inputParams' can't be found in a schema class 'SOAP::Serializer'</code> Seen with SOAP::Lite releases 0.65, 0.66 and 0.68. These releases do not generate the required client code correctly from the WSDL and thus cannot encode complex types correctly. |
| | * <code>Can't bless non-reference value at WSPSIBlast.pm line 127.</code> This is a server exception indicating that the contents of the message sent to the server could not be decoded correctly. Currently this has only been observed when using SOAP::Lite 0.710.10. | | * <code>Can't bless non-reference value at WSPSIBlast.pm line 127.</code> This is a server exception indicating that the contents of the message sent to the server could not be decoded correctly. Currently this has only been observed when using SOAP::Lite 0.710.10. |
| | | | |
| | * [[http://search.cpan.org/~mkutter/SOAP-Lite-0.710.08/|SOAP::Lite 0.710.08]] | | * [[http://search.cpan.org/~mkutter/SOAP-Lite-0.710.08/|SOAP::Lite 0.710.08]] |
| | | | |
| - | ===== Java: Axis 1.3/1.4 does not work? ===== | + | To determine the installed version of SOAP::Lite: |
| | + | <code csh>perl -e 'use SOAP::Lite; print $SOAP::Lite::VERSION, "\n";'</code> |
| | + | |
| | + | ===== Java: Axis 1.2/1.3 does not work? ===== |
| | | | |
| - | The client stubs produced by Axis WSDL2Java are version specific. The supplied clients have been built using Axis 1.2, provided in [[http://www.ebi.ac.uk/Tools/webservices/downloads/java/lib.zip|lib.zip]]. If you want to use a different version of Axis you will have to regenerate the stubs and re-compile, see the [[tutorials:java|Java tutorial]] for further details. | + | The client stubs produced by Axis WSDL2Java are version specific. The supplied clients have been built using Axis 1.4, provided in [[http://www.ebi.ac.uk/Tools/webservices/downloads/java/lib-1.4.zip|lib-1.4.zip]]. If you want to use a different version of Axis you will have to regenerate the stubs and re-compile, see the [[:tutorials:06_Programming:Java|Java tutorial]] for further details. |
| | | | |
| | ===== Java: NoClassDefFoundError error ===== | | ===== Java: NoClassDefFoundError error ===== |
| | * ''-cp .'' adds the current directory to the Java classpath | | * ''-cp .'' adds the current directory to the Java classpath |
| | | | |
| - | If you are using the provided libraries from ''[[http://www.ebi.ac.uk/Tools/webservices/downloads/java/lib.zip|lib.zip]]'' and did not uncompress the ''zip'' file in your current directory, you will need to specify the full path to the folder. | + | If you are using the provided libraries from ''[[http://www.ebi.ac.uk/Tools/webservices/downloads/java/lib-1.4.zip|lib-1.4.zip]]'' and did not uncompress the ''zip'' file in your current directory, you will need to specify the full path to the folder. |
| | | | |
| | ===== Taverna: How can I poll for my results? ===== | | ===== Taverna: How can I poll for my results? ===== |
| | | | |
| - | To iterate over different sequences, as Taverna has no "for" structures, you can do the following: | + | ==== Taverna 1.x ==== |
| - | | + | |
| - | 1) Create a nested workflow and open it for editing | + | |
| - | | + | |
| - | 2) In the nested workflow you need to create a workflow input called JobID, no outputs are required although you can create an output called JobID as well, it might make things more convenient later. | + | |
| - | | + | |
| - | 3) You need to put three things in the nested workflow: | + | |
| - | | + | |
| - | a) The process that checks the status given an ID, link this with the JobID input | + | |
| - | | + | |
| - | b) An operation (probably a beanshell) which takes the output from (a) and returns true if and only if the output means that there are some results available. So, if the status as a string from (a) is 'SUCCESS' | + | |
| - | when there are results you create a beanshell with a single string input called 'state' and a single string output called 'finished' and have the script as 'finished = (state.equals("DONE"));', this will return | + | |
| - | 'true' if the status was DONEand false otherwise. | + | |
| - | | + | |
| - | c) A 'Fail if false' operation linked to the output of (b) | + | |
| | | | |
| - | 4) In the nested workflow you must check the 'critical' checkbox for the 'Fail if false' processor. This means that if this fails (i.e. if the input is false) the entire nested workflow will fail. | + | Polling is implemented using the retry mechanism and a nested workflow. See [[:tutorials:07_workflows:taverna#polling_for_results|Taverna tutorial: Polling for Results]]. |
| | | | |
| - | 5) In the top level workflow you now need to configure the retry behaviour for the nested workflow. Exactly what you put here depends a bit on how long you expect the job to take but I suggest keeping the backoff factor very close to 1.0 (or leaving it at 1.0, i.e. the delay between retries remains the same). Set the retry delay to 30000 for a 30 seconds interval and, say, 10 retries. This would mean the status would | + | Example Taverna 1.x workflows for the EMBL-EBI services can be found in [[http://www.myexperiment.org/|myExperiment]]. |
| - | be polled for 5 minutes or until it succeeded. | + | |
| | | | |
| - | 6) Again in the top level workflow you'll need a process that gets the results from a completed job given the JobID. This process can only be allowed to run when the nested worklow created in (1) has succeeded, fortunately you can create a coordination link to it (right click on the processor, select 'coordinate from' and choose the nested workflow). | + | ==== Taverna 2.x ==== |
| | | | |
| - | You'll obviously also need to pass in the JobID. If you previously added a 'pass through' JobID output to the nested workflow you don't need the coordination link, it's really a matter of personal preference which way you do it. | + | In Taverna 2 the polling loop is handled as a property of the processor, rather then as an external workflow. |
| | | | |
| - | Although this may sound slightly long winded it's actually quite quick to do. Basically you're using Taverna's retry policy control to repeatedly poll the state, as Taverna will only retry something that failed you have to make any state other than 'results ready' into a failure condition which is what the nested workflow described above will do. | + | Example Taverna 2 workflows for the EMBL-EBI services can be found in [[http://www.myexperiment.org/|myExperiment]]. |
| | | | |
| | ===== Which databases are available? ===== | | ===== Which databases are available? ===== |
| | | | |
| - | For a list of the database names used by [[services:dbfetch|dbfetch]] and the sequence similarity search services (e.g. [[services:fasta|WSFasta]], [[services:ncbiblast|WSNCBIBlast]] and [[services:wublast|WSWUBlast]]) see: [[help:databases|Available databases]]. | + | For a list of the database names used by [[:services:dbfetch|dbfetch]] and the sequence similarity search services (e.g. [[services:fasta|WSFasta]], [[:services:ncbiblast|WSNCBIBlast]] and [[:services:wublast|WSWUBlast]]) see: [[:help:databases|Available databases]]. |
| | | | |
| - | Some of the services provide methods to get the details of the available databases at run time, see the page for the service for further details ([[about:webservices|list]]). | + | Some of the services provide methods to get the details of the available databases at run time, see the page for the service for further details ([[:|list]]). |
| | | | |
| | ===== How do I cite a service? ===== | | ===== How do I cite a service? ===== |
| | | | |
| - | Depending on the service there may be a specific publication for the service which should be used, see the service documentation for details ([[about:webservices|list]]). If the service does not have a specific publication then one of the generic publications detailing the EBI Web Services can be used, see [[about:references|references]]. | + | Depending on the service there may be a specific publication for the service which should be used, see the service documentation for details ([[:|list]]). If the service does not have a specific publication then one of the generic publications detailing the EMBL-EBI Web Services can be used, see [[:about:references|references]]. |
| | | | |
