[ Solr ] Error Document is missing mandatory uniqueKey field id

In the schema.xml file, It is mentioned id as required field = true.

Also the document that we try to index in SOLR do not contain this id field and hence SOLR throws this error.

Solution

1. Either add id to all your documents

OR

2. Remove required = true form schema file for id field.

Hope This Helps!!!!

[Google Search Appliance ] Migrate Autosuggestions from one production GSA Box to Another Production GSA Box

We all know that Google Search Appliance learns all the search queries that are getting executed and then it generates the Autosuggestion list.

However If you want to migrate all autosuggestions from one production box to another then there is no feature currently available to achieve this task. 

There is a feature request # 4411407 for allowing import query suggestion into GSA box.

However I wanted to achieve this by some custom means So I followed the below steps and it worked perfectly fine.

1) Export the suggestion in text file from first GSA Box.

2) Write a script to read each line from that files and execute the search query against word /phrase to new GSA Box.

3) This will create a new autosuggestion list in new GSA Box.

Hope this Helps!!!

[Google Search Appliance] Semantic Search Support

Below features can be configured in GSA to enable Semantic Search

1. Query Expansion
2. Related Search
3. KeyMatch Search

All the above features works on a predefined list/set of words. In order to implement these features we have to configure the list into GSA.

1)   Query Expansion

Query expansion enables GSA to automatically add extra terms to a user's search query, in order to return additional relevant results. 

When query expansion is enabled, the appliance can expand two types of terms:

Stem Words: Words that share the same word stem as the word given by the user. For example, if a user searches for "engineer," the appliance could add "engineers" to the query. (Fast, Faster, Fastest)

Terms of one or more space-separated words that are synonymous or closely related to the words given by the Business user.

For example, if a user searches for "FAQ," the appliance could add "frequently asked questions" to the query

For example if recruiter searches for “P1” the appliance could add “priority 1” to the query.

Query Expansion feature can be implemented by configuring the Synonym file into GSA.

Rule	Meaning	Comments
Java = Spring	Bi-Directional Synonym	If user searches for Java, then user will get results for Java as well as Spring. If user searches for Spring, then user will get results for Java as well as Spring.
Java > Spring, Java > Hibernate	One-Directional Synonym	If user searches for Java, then user will get results for Java as well as for Spring and Hibernate. However If user searches for Spring, then user will only get result for Spring and same applies for Hibernate
{Java, Spring, Hibernate]	Bi-Directional Synonym	If user searches either of the words (Java, Spring, Hibernate), then user will get results for all the search terms.

Pros: If proper and accurate Synonym file is configured in GSA then it will enhanced the user search experience.

Cons: Improper Synonym file can result into false positive search results being returned.

2)  Related Search

Related Search feature can be used to associate alternative words or phrases with specified search terms. When a user enters the specified search term, the alternative word or phrase appears as a suggestion. The user can click on the suggested alternative to start another search.

Search Term	Related Query
P1	Priority 1

It works as the replacement of original “Search Term” with “Related Query”

3)  KeyMatch Search

For a given search term or phrase search; GSA can return a specific set of results to user.

Search Term	Type	Url of Match	Title of Match
GSA	Keyword Match	http://tinyurl.com	Search Architect for Google Search and Match Project

ELASTICSEARCH JDBC CONNECTOR

ELASTICSEARCH JDBC CONNECTOR

1) Check the dependency and download appropriate JDBC connector version code as per your elasticsearch version.

https://github.com/jprante/elasticsearch-jdbc#recent-versions

2) Download the code from below location:

https://github.com/jprante/elasticsearch-jdbc

3) Compile the code using “mvn clean install”. It will create a jar file in target folder.

4) Copy the jar file into elasticsearch lib folder.

5) Download the JDBC importer distribution from below location

http://xbib.org/repository/org/xbib/elasticsearch/importer/elasticsearch-jdbc/<version>/elasticsearch-jdbc-<version>-dist.zip

6) unpack

unzip elasticsearch-jdbc-<version>-dist.zip

7) This will be our $JDBC_IMPORTER_HOME

8) Set the below path 

#Elastic Search

$> ES_HOME=/Users/jayeshbhoyar/Desktop/Programs/elasticsearchinstance/

$> export ES_HOME

# JDBC_IMPORTER_HOME

$> JDBC_IMPORTER_HOME=/Users/jayeshbhoyar/Desktop /Programs/elasticsearchinstance/elasticsearchjdbc1.5.2.0/

$> export JDBC_IMPORTER_HOME

9) Download the JDBC driver jar file from vendor site and copy it in “ES_HOME/lib” and “JDBC_IMPORTER_HOME/lib” folders

10) Go to “JDBC_IMPORTER_HOME” Folder and execute the below command:

bin=$JDBC_IMPORTER_HOME/bin 

lib=$JDBC_IMPORTER_HOME/lib 

echo '{     

"type" : "jdbc",     

"jdbc" : {         

"url" : "jdbc:mysql://localhost:3306/test",         

"user" : "",         "

password" : "",         

"sql" : "select *, id as _id from orders"     

} 

}' | java \        

-cp "${lib}/*" \        

-Dlog4j.configurationFile=${bin}/log4j2.xml \        

org.xbib.tools.Runner \        

org.xbib.tools.JDBCImporter

It will start pulling the records from orders table and insert it into elasticsearch.

Sample File:

{

    "type" : "jdbc",

    "bulk_size" : 1000,

"max_bulk_requests" : 50,

"bulk_flush_interval" : "10s",

"max_concurrent_bulk_requests" : 16,

"max_bulk_actions" : 1000,

    "jdbc" : {

        "url" : "jdbc:oracle:thin:@hostname:port:SID",

        "driver": "oracle.jdbc.OracleDriver",

        "user" : "username",

        "fetchsize" : 1000,

        "password" : "password",

        "sql" : "SELECT * FROM ABC”,

        "index" : "INDEXNAME",

        "type" : "INDEXTYPE",

        "index_settings": {

            "index": {

                "refresh_interval": "5s"

            }

        } ,

        "max_bulk_actions" : 1000,

    "metrics" : {

        "enabled" : true,

        "interval" : "1m",

        "logger" : {

            "plain" : true,

            "json" : false

        }

    },

        "max_concurrent_bulk_requests" : 16,

        "connection_properties" : {

            "oracle.jdbc.TcpNoDelay" : false,

            "useFetchSizeWithLongColumn" : false,

            "oracle.net.CONNECT_TIMEOUT" : 10000,

            "oracle.jdbc.ReadTimeout" : 50000

        },

        "elasticsearch" : {

         "cluster" : "CLUSTERNAME",

         "host" : "ELASTICSEARCHHOSTNAME",

         "port" : 9300

    }

    }

}

For more settings and more information visit below link.

https://github.com/jprante/elasticsearch-jdbc

Hope this Helps!!!!

Jayesh Bhoyar

Reference:

https://github.com/jprante/elasticsearch-jdbc

Check More Articles on ElasticSearch:
http://technical-fundas.blogspot.com/2015/07/elasticsearch-bigdesk-plugin.html

ElasticSearch BigDesk Plugin

ElasticSearch BigDesk Plugin

Bigdesk Plugin makes it very easy to see how your Elasticsearch cluster is doing. 
Just install it as an Elasticsearch plugin, download locally or run online from the web, 
then point it to the Elasticsearch node REST endpoint and have fun.

To install Bigdesk as an Elasticsearch plugin on a particular Elasticsearch node, 
navigate to the node installation folder i.e. ES_HOME and run the following command:

bin/plugin -install lukas-vlcek/bigdesk


Then you can navigate your web browser to 
http://HOSTNAME:PORT/_plugin/bigdesk/

Example:
http://localhost:9200/_plugin/bigdesk/

Reference:

http://bigdesk.org/

Check More Articles on Elasticsearch:

http://technical-fundas.blogspot.com/2015/07/elasticsearch-jdbc-connector.html

http://technical-fundas.blogspot.com/2014/08/elasticsearch-install-and-access-head.html

Explaining Iaas, Paas, SaaS

Very well explained IaaS/PaaS/SaaS with a simple Pizza example.

No matter which service we use; ultimately we have to pay for the Service that we are using.

[elasticsearch] Install and access head plugin in elasticsearch

There are many customized plugins that comes with elasticsearch. In this article we will see how to install and then how to use the head plugin with elasticsearch.


Here are few steps that we need to follow:

Step1:
To install head plugin from site use following command:

bin/plugin --install mobz/elasticsearch-head

 You will be able to install the plugin using above only when you have internet connectivity and your firewall alows you to download and install

Else you can manually download the elasticsearch-head.zip plugin from https://github.com/mobz/

Run the following command

bin/plugin --url file:///d:\elasticsearch-head.zip --install mobz/elasticsearch-head

Step 2:

Restart the elasticsearch instance and you will be able to access it as

http://localhost:9200/_plugin/head/