Starting a Search

The initialization of substructure searching is similar to duplicate searching , but the JChemSearchOptions object needs to be created with SearchConstants.SUBSTRUCTURE constant value.
Java example:

    ...                                  // Initialize connection

    String mol = "[*]c1cccc([Cl,Br])c1"; // Query structure
    String structureTableName = "cduser.structures";

    JChemSearch searcher = new JChemSearch(); // Create searcher object
    searcher.setQueryStructure(mol);
    searcher.setConnectionHandler(connHandler);
    searcher.setStructureTable(structureTableName);
    JChemSearcOptions searchOptions = new JChemSearchOptions( SearchConstants.SUBSTRUCTURE );
    searcher.setSearchOptions(searchOptions);
    searcher.run();

More examples:

• JSP: <JChem's home>/examples/db_search/searching.jsp

• ASP: <JChem's home>/examples/asp/searching.asp

Running Search in a Separate Thread

Since substructure searching can be time consuming, it is reasonable to create a new thread for the search. If the runmode property of JChemSearch is set to JChemSearch.RUN_MODE_ASYNCH_COMPLETE , then searching runs in a separate thread.

The progress of the search can be checked by the following properties of JChemSearch :

Java application example:

    searcher.setRunMode(JChemSearch.RUN_MODE_ASYNCH_COMPLETE);
    searcher.run();
    while( searcher.isRunning() ) {
        String msg =  searcher.getProgressMessage() ;
        int count =  searcher.getResultCount() ;
        ...                 // Displaying
        Thread.sleep(1000);
    }

Please, see the AsyncSearchExample.java example in the examples/java/search directory.
JSP example:

 if(searcher.isRunning()) {
        %>
        <p>Please wait. Searching.....
        <p><%= searcher.getProgressMessage() >%
        <p>Hits: <%= searcher.getResultCount() %>
        <p>
        <script LANGUAGE="JavaScript">
        <!--
        window.setTimeout('window.location.reload(false)', 5000);
        //-->
        </script>
        >%
    }

More examples:

• JSP: <JChem's home>/examples/db_search/searching.jsp

• ASP: <JChem's home>/examples/asp/searching.asp

Retrieving Results

If the resultTableMode property of JChemSearch is set to JChemSearch.NO_RESULT_TABLE , then the following properties can be used for retrieving the results:

The two ways of retrieving the results of the search are:

• directly accessing the results from the JChemSearch object, or

• preparing a SQL statement and accessing the results from the Connection object.

Retrieving Results from the JChemSearch object

The process of retrieving the results of the search from the JChemSearch object is based on the getHitsAsMolecules(. . .) and the getHitsRgDecomp(. . .) methods. In both cases the same JChemSearch object is needed that was used to run the search.

• Using the getHitsAsMolecules(...) method

  • Apply the array of cd_id values obtained by a JChemSearch.getResults() or JChemSearch.getResult(int) call.

  • Create and configure a HitColoringAndAlignmentOptions object.
    Generally, the following properties are set:

    coloring	Specifies if substructure hit coloring should be used.
    enumerateMarkush	Specifies if markush structures should be hit enumerated according to the query structure.
    removeUnusedDef	Specifies whether unused R group definitions should be removed from the results.
    alignmentMode	Specifies what form of alignment to use for hit display. The following values are accepted: ALIGNMENT_OFF (default), ALIGNMENT_ROTATE, ALIGNMENT_PARTIAL_CLEAN

If this options object is null, the molecules will be returned in their original form.

• • Create an ArrayList of the data field names that should be returned. If this ArrayList is empty, no values will be returned.

  • Create an empty ArrayList to hold the fetched data field values. The ArrayList will have Object array elements, one for each molecule. Inside each Object array will be the fetched data field values.

  • To display the molecules, use the Molecule array return value.
    Java example:

            JChemSearch searcher = new JChemSearch();       // create searcher object
            // ...                                          // run search
    
            int[] resultIds = searcher.getResults();
    
            HitColoringAndAlignmentOptions displayOptions = new HitColoringAndAlignmentOptions();
            displayOptions.setColoringEnabled(hitsShouldBeColored);
            if (hitsShouldBeRotated) {
                displayOptions.setAlignmentMode(HitColoringAndAlignmentOptions.ALIGNMENT_ROTATE);
            } else {
                displayOptions.setAlignmentMode(HitColoringAndAlignmentOptions.ALIGNMENT_OFF);
            }
            // ...
    
            List<String> dataFieldNames = new ArrayList<String>();
            dataFieldNames.add("cd_id");
            dataFieldNames.add("cd_formula");
            dataFieldNames.add("cd_molweight");
    
            List<Object[]> dataFieldValues = new ArrayList();
    
            Molecule[] mols = searcher.getHitsAsMolecules(resultIds, displayOptions,
                    dataFieldNames, dataFieldValues);
            // ...

    More examples:

  • JSP: <JChem's home>/examples/db_search/searchresults.jsp

• Using the getHitsAsRgDecomp(...) method

  • Apply the array of cd_id values obtained by a JChemSearch.getResults() or JChemSearch.getResult(int) call.

  • Set the attachmentType parameter to one of the ATTACHMENT_... constants of the RGroupDecomposition class (the parameter value in most cases is RGroupDecomposition.ATTACHMENT_POINT).

  • Retrieve results by using methods of RgDecompResults
    Java example:

            JChemSearch searcher = new JChemSearch();       // create searcher object
            // ...                                          // run search
    
            int[] resultIds = searcher.getResults();
            RgDecompResults rgdResults = searcher.getHitsAsRgDecomp(resultIds, RGroupDecomposition.ATTACHMENT_POINT);
    
            // Decomposition objects corresponding to the targets
            Decomposition[] decompositions = rgdResults.getDecompositions();
    
            // Covering Markush structure of all hits
            RgMolecule markush = rgdResults.getHitsAsMarkush();
    
            // R-group decomposition table of hits
            Molecule[][] rgdTable = rgdResults.getHitsAsTable();

Retrieving Results using a SQL statement

The process of retrieving the results of the search from the ResultSet Object:

• Use SQL statements like

      SELECT cd_structure, ...
      FROM cduser.structures
      WHERE cd_id=12532

• Apply the cd_id value obtained by a JChemSearch.getResult(...) call in the condition of the SQL statement

To display the molecule, use cd_structure obtained from the ResultSet.
In the case of web applications

• Use MarvinView to display the data. You can use

  • MarvinView tables (one applet per page)

  • HTML tables (several MarvinView applets per page)

  Use the JavaScript routines in marvin.js to make sure that the pages can be viewed in different browsers, GUI-s (AWT/Swing), and JVM-s (built-in/Java plugin).

• Modify values from the cd_structure column using HTMLTools.convertForJavaScript(...) for inserting into a web page in an applet parameter.
  Java example:

          int[] cdIds = searcher.getResults();
  
          String retrieverSql = "SELECT cd_molweight from " + structTableName
                  + " where cd_id = ?";</b>
          PreparedStatement ps = connectionHandler.getConnection()
                  .prepareStatement(retrieverSql);
          try {
              for (int i = 0; i < cdIds.length; i++) {
                  int cdId = cdIds[i];
                  ps.setInt(1, cdId);
                  ResultSet rs = ps.executeQuery();
                  if (rs.next()) {
                      System.out.println("Mass: " + rs.getDouble(1));
                  } else {
                      ; // has been deleted in the meantime?
                  }
              }
          } finally {
              ps.close();
          }

  Please, see the RetrievingDatabaseFieldsExample.java example in the examples/java/search directory.
  Please, see MultipleQueriesExample.java demonstrating two approaches for calculating the intersection of the result of multiple queries.
  JSP example for displaying search results on a web page by a JSP script:

      ...
      Hits: <%= searcher.getResultCount() %> 
      <font size="1">
      <%= searcher.isMaxResultCountReached()? " (maximum hits reached)" : "" %>
      <%= searcher.isMaxTimeReached()? " (maximum time reached)" : "" %>
      </font>
      <center>
      <script LANGUAGE="JavaScript1.1" SRC="../../../marvin/marvin.js">
      </script>
      <script LANGUAGE="JavaScript1.1">
      <!--
      mview_name="mview";
      mview_begin("../../../marvin",
              "<%= cols*cellWidth+cols-1 %>",
              "<%= rows*cellHeight+rows-1 %>");
      mview_param("rows", "<%= rows %>");
      mview_param("cols", "<%= cols %>");
      mview_param("navmode", "rot3d");
      mview_param("molbg", "#000000");
      mview_param("bgcolor", "#e0e0e0");
      mview_param("border", "1");
      mview_param("animate", "0");
      mview_param("layout0", ":4:1:"+
          "L:0:0:1:1:w:n:0:10:"+ <%/* ID              */%>
          "L:1:0:1:1:w:n:0:10:"+ <%/* Stock           */%>
          "M:2:0:1:1:c:n:1:10"); <%/* Molecule        */%>
      mview_param("param0", ":"+
          "L:11b:"+
          "L:10:"+    
          "M:<%= structureWidth %>:<%= structureHeight %>");
      <%
      /*
       * Writing data into the cells in the applet
       */
      int cellIndex=0;
      for(int i=start; i<start+cells; i++) {
          int id = searcher.getResult(i);
          /*
           * SQL statement for retrieving structures
           */
          String sql =
                  "SELECT " +
                  structureTableName + ".cd_id, "+
                  structureTableName + ".cd_structure, " +
                  stockTableName + ".quantity\n" +
                  "FROM "+ structureTableName + ", " +
                  stockTableName + "\n" +
                  "WHERE " +
                  structureTableName + ".cd_id = " +
                  stockTableName + ".cd_id AND " +
                  structureTableName + ".cd_id = " + id;
          Statement stmt = con.createStatement();
          ResultSet rs = stmt.executeQuery(sql);
          try {
              if(rs.next()) {
                  String dbMolfile = new String(DatabaseTools.readBytes(rs, 2),"ASCII");
                  float dbQuantity = rs.getFloat(3);
                  %>
                  mview_param("cell<%= cellIndex %>",
                      "|ID: <%= id %>"+
                      "|<%= Math.round(dbQuantity) %> mg"+
                      "|<%= HTMLTools.convertForJavaScript(
                          dbMolfile) %>");
                  <%
              }
          } finally {
              rs.close();
              stmt.close();
          }
          cellIndex++;
      }
      %>
      mview_end();
      //-->
      </script>
      </center>
      ...

  Click here to see the live result of the above code.

  More JSP examples:

• <JChem's home>/examples/db_search/searchresults.jsp

• <JChem's home>/examples/simple_db_search/searchresults.jsp

To store the results in a table, the name of the table should be specified by the resultTable property of JChemSearch , and also the resultTableMode property should be set to either JChemSearch.CREATE_OR_REPLACE_RESULT_TABLE or JChemSearch.APPEND_TO_RESULT_TABLE .

Retrieving hits as soon as they are found

If the runmode property of JChemSearch is set to JChemSearch.RUN_MODE_ASYNCH_PROGRESSIVE , then searching runs in a separate thread and hits can be retrieved as soon as they are found. Note that this mode does not support any ordering:

    List<int[]> hitsByPages = new ArrayList<int[]>();
    int[] nextPage = new int[NUMBER_OF_HITS_PER_PAGE];
    int idxForNextPage = 0;
     *searcher.setOrder(JChemSearch.NO_ORDERING)* ;
     *searcher.setRunMode(JChemSearch.RUN_MODE_ASYNCH_PROGRESSIVE)* ;
     *searcher.run()* ;
    while ( *searcher.hasMoreHits()* ) {
        nextPage[idxForNextPage++] =  *searcher.getNextHit()* ;
        if (idxForNextPage == NUMBER_OF_HITS_PER_PAGE) {
            synchronized (hitsByPages) {
                hitsByPages.add(nextPage);
                hitsByPages.notifyAll(); // notify any who may be in wait for the next page
                nextPage = new int[NUMBER_OF_HITS_PER_PAGE];
                idxForNextPage = 0;
            }
        }
    }

    // Hits for the last page if any
    if (idxForNextPage > 0) {
        int[] lastPage = new int[idxForNextPage];
        System.arrayCopy(nextPage, 0, lastPage, 0, idxForNextPage);
        synchronized (hitsByPages) {
            hitsByPages.add(lastPage);
            hitsByPages.notifyAll(); // notify any who may be in wait for the next page
        }
    }

Caching Structures

To boost the speed of substructure searching, JChem caches fingerprints and structures in the searcher application's memory. For more information, see the JChem database concepts section.

Combining SQL queries with Structure Searching

Many times structure information is only one of several conditions that a complex query has to check. In those cases structure searches should be combined with SQL queries.
Example: Suppose that quantities on stock are stored in a table different from the structure table. We are querying compounds that contain a given substructure and their quantity on stock is not less than a given value.
Two ways of performing the combined query:

• Structure Searching Followed by SQL Query

  • Initialize a JChemSearch object.

  • Instruct JChemSearch to save the cd_id values of found compounds: set the name of the result table using the setResultsTable method.

  • Run structure searching.

  • Query the join of the structure table, the result table, and the stock table using an SQL SELECT statement. (See the syntax of SELECT in the documentation of your database engine.) Example:

    SELECT cd_structure, quantity FROM hits, structures, stock
        WHERE hits.cd_id=structures.cd_id AND stock.cd_id=structures.cd_id AND stock.quantity < 2

• SQL Query Followed by Structure Searching
  An arbitrary SQL query can be specified as a filter for the filterQuery property. The (first) result column should contain the allowed cd_id values.
  Java example:

          JChemSearchOptions jcSearchOptions = new JChemSearchOptions(SearchConstants.SUBSTRUCTURE);
          jcSearchOptions.setFilterQuery("SELECT cd_id FROM " + stockTableName
                  + " WHERE quantity < 2");

  Please, see the SearchWithFilterQueryExample.java example in the examples/java/search directory.

Calculated columns

Databases of chemical structures can contain various calculated values. These are specified upon table creation using Chemical Terms expressions and their value is calculated during database import. Please, see how calculated columns are stored in JChem tables. While executing database search these fields can be considered. Assuming that the table "search_example" exists with the columns logp, rtbl_bnd_cnt and pka_ac_2 a search can be executed as follows:

        String[] columns = { "logp", "rtbl_bnd_cnt", "pka_ac_2" };
        double[] thresholds = { 3.85, 3, 18 };
        for (int i = 0; i < columns.length; i++) {
            searchOptions.setFilterQuery("SELECT cd_id FROM " + tableName
                    + " WHERE " + columns[i] + ">" + thresholds[i]);
            jcs.setSearchOptions(searchOptions);
            jcs.run();
            ... // handling results
        }

Please, see how to create calculated columns with JChem Manager.

Using command line tool the following command performs the same operation:

        jcman c search_example
            --coldefs ", logp float8, rtbl_bnd_cnt float8, pka_ac_2 float8"
            --ctcolcfg "logp=logp(); rtbl_bnd_cnt=rotatableBondCount()>4; pka_ac_2=pKa(\"acidic\",\"2\")"

To an existing table, where the appropriate columns have been created, calculated columns can be added using jcman tool:

        jcman m search_example
            --add-ctcolcfg "logp=logp(); rtbl_bnd_cnt=rotatableBondCount()>4; pka_ac_2=pKa(\"acidic\",\"2\")"

Please, see CalculatedColumnsSearchExample.java demonstrating the usage of calculated columns during search.

Chemical Terms filtering

Calculated values do not need to be stored in a database field. If they should be used for a temporary filtering, they may be calculated "on the fly". These are specified for database search using the setChemTermsFilter option:

        searchOptions.setChemTermsFilter("pka(h(0))> 2");

Please, see SortedSearchExample.java demonstrating hits ordering.
Please, see SearchWithFilterQueryExample.java demonstrating how to filter search results based on other (possibly) database tables.

Setting More Properties

There are several other properties that modify the behavior of JChemSearch , like

        JChemSearchOptions searchOptions = new JChemSearchOptions(SearchConstants.SIMILARITY);
        searchOptions.setDissimilarityThreshold(0.6f);
        // ...
        JChemSearch searcher = new JChemSearch();
        searcher.setStructureTable(structTableName);
        searcher.setQueryStructure("c1ccccc1N");
        searcher.setSearchOptions(searchOptions);
        searcher.setConnectionHandler(connectionHandler);
        // Change the default ordering (which is by similarity and id)
        searcher.setOrder(JChemSearch.ORDERING_BY_ID);

Similarity Searching With Molecular Descriptors

Users can open up new ways of similarity searching by using a number of built-in molecular descriptor types other than the default chemical hashed fingerprints. There are a number of built-in molecular descriptors available, including CF, PF, Burden eigenvalue descriptor (or BCUTTM) and various scalar descriptors.
The following example shows how simple it is to setup molecular descriptors for your compound library. The first command creates a table called compound_library and the second command adds the molecules from an sdf file. The third command uses the 'c' option to create the molecular descriptor with the name of the structure table set by the -a flag, and the chemicalfingerprint descriptor type set by the -k flag. The command omits the database login information that was stored previously with the -s option. See the jcman command options and the GenerateMD command options for more information. Creating and assigning molecular descriptors to database structure tables is discussed with the GenerateMD command.

        jcman c compound_library
        jcman a compound_library my_compound_group.sdf
        generatemd c -a  *compound_library*  -k CF  chemical_fingerprint 

Below is an example that runs the similarity search with the new chemical fingerprint. The molecular descriptor name, chemical_fingerprint , is set as a search option and the similarity search is run normally:

        JChemSearch searcher = new JChemSearch(); // Create searcher object
        searcher.setQueryStructure(mol);
        searcher.setConnectionHandler(connHandler);
        searcher.setStructureTable("compound_library");
        JChemSearcOptions searchOptions = new JChemSearchOptions(JChemSearch.SIMILARITY);
        searchOptions.setDescriptorName("chemical_fingerprint"); 
        searchOptions.setDissimilarityThreshold(0.2f);
        searcher.setSearchOptions(searchOptions);
        searcher.run();

Molecular Descriptor Configuration Options

Application end-users may need further information about the molecular descriptors to select an appropriate molecular descriptor for their search. Application developers can extract this information from the database and display it to the end-user to help with selection. In this Java example, a MDTableHandler is created using the database connection and the name of the structure table. The MDTableHandler provides access to the Molecular Descriptors and the embedded configurations, metrics, and default dissimilarity thresholds.

        // Start with database connection handler and name of the structure table
        MDTableHandler mdth = new MDTableHandler(connectionHandler, structureTableName);
        String[] descriptorIds = mdth.getMolecularDescriptors();
        for (int x = 0; x < descriptorIds.length; x++){
            String mdName = descriptorIds[x];
            MolecularDescriptor descriptor = mdth.createMD(mdName);
            // Ge descriptor names
            String descriptorName = descriptor.getName();
            // Get descriptor comments
            String descriptorComment = mdth.getMDComment(mdName);
            // Get available metrics for each configuration
            String[] configNames = mdth.getMDConfigs(mdName);
            for (int i = 0; i < configNames.length; i++) {
                MolecularDescriptor tempDescr = (MolecularDescriptor) descriptor.clone();
                String config = mdth.getMDConfig(mdName,configNames[i]);
                tempDesc.setScreeningConfiguration(config);
        
                // Get metric name
                String metricName = tempDesc.getMetricName();
                // Get default thresholds
                String defaultThreshold = tempDesc.getThreshold();
                // ...
            }
            // Display code can go here
            // ...
        }

After selecting a molecular descriptor and other desired parameters, such as the descriptor configuration and the metric, the custom molecular descriptor name is set as a search option and the similarity search is run as normal. If the descriptor name, configuration or metric is omitted, a stored default value is used.

        JChemSearch searcher = new JChemSearch(); // Create searcher object
        searcher.setQueryStructure(mol);
        searcher.setConnectionHandler(connHandler);
        searcher.setStructureTable(structureTableName);
        JChemSearcOptions searchOptions = new JChemSearchOptions(JChemSearch.SIMILARITY);
        searchOptions.setDescriptorName(selectedDescriptor); 
        searchOptions.setDescriptorConfig(selectedConfig); 
        searchOptions.setDissimilarityMetric(selectedMetric); 
        searchOptions.setDissimilarityThreshold(0.8f);
        searcher.setSearchOptions(searchOptions);
        searcher.run();

More examples:

• Dissimilarity Example using MDTableHandler

Customizing the Molecular Descriptor

In addition, a cheminformatics expert can generate and fine tune a custom made molecular descriptor. Further information on generating the custom molecular descriptors can be found here .

Please, see SimilaritySearchExample.java demonstrating how descriptors are generated and similarity searches executed on them.