Installation

Java

OpenJDK 17 Installation

Download the Setup from the Adoptium Website OpenJDK Latest release, download the "Install JDK" Installer. (for example: OpenJDK17U-jdk_x64_windows_hotspot_17.0.5_8.msi)

Before starting the installation, make sure that no Oracle Java installation is present! If there is one or more, uninstall all of them first!

MSI started with a double click:

Click on "Next"

Add the Feature "Set JAVA_HOME environment variable" and click "Next"

Click on "Install"

If the installation was successful

Elasticsearch

Elasticsearch 7.17.15 Installation

ElasticSearch must be installed by using the ElasticSearch .zip archive. This file is included in the inPoint.Wizard under "files\ElasticSearch". There is no possibility to install ElasticSearch with a MSI file.

The service creation and configuration can be achieved through the elasticsearch-service.bat. It can be used for install/remove/start/stop the ElasticSearch service. This batch file is based on the Apache Commons Daemon project. All settings which are needed for the ElasticSearch service must be configured with the Environment variables before running this batch file. The service must be reinstalled if there are changes inside the environment variables. Otherwise the new variables will be not used.

To make changes after the service creation - you can use the Manager GUI by executing the following cmd: "bin\elasticsearch-service.bat manager ElasticSearch" (The last parameter is the %service id% - default is elasticsearch-service-x64)

As example:

Steps to install ElasticSearch:

• Get the zip file from: https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.17.15-windows-x86_64.zip

• Extract it to (as example): C:\Program Files\Elastic\ElasticSearch\7.17.15

  • Set the following machine environment variables:
    • set ES_HOME to the current ElasticSearch install path (Example: C:\Program Files\Elastic\ElasticSearch\7.17.15)
    • set ES_JAVA_HOME to the current Java installation directory (Example: C:\Program Files\Java\jdk-17.0.5+8) (If you update this value afterwards then there is no need to reinstall the service)
    • set ES_PATH_CONFIG to the config directory (Example: C:\Program Files\Elastic\ElasticSearch\7.17.15\config) (This contains the elasticsearch.yml)

• Modify the ElasticSearch.yml:

  • Set the following properties (based on your needs):
    • cluster.name: elasticsearch
    • path.data: D:\Elastic\Data
    • path.logs: D:\Elastic\Logs
    • network.host: %COMPUTERNAME%
    • http.port: 9200

• Install ElasticSearch as a service (open CMD):

  • Go to the $ES_HOME$ folder
  • Execute: "\bin\ElasticSearch-Service.bat install ElasticSearch" (the last parameter is the %service id% - the inPoint.Wizard is using "ElasticSearch" - the default "elasticsearch-service-x64" will be used if u keep it empty)
  • Execute: net start ElasticSearch (inside CMD)

• Check if ElasticSearch is running:

To quickly check if Elasticsearch is running, you can use Chrome or Firefox to call this URL (Internet Explorer works as well, but it will download the result as text-file instead of showing it directly):

Example: http://<host>:9200

Result should be:

{
  "name" : "<host>",
  "cluster_name" : "<host>",
  "cluster_uuid" : "bsfLa0aLR76SK6NtvswW5A",
  "version" : {
    "number" : "<version>",
    "build_flavor" : "default",
    "build_type" : "zip",
    "build_hash" : "120eabe1c8a0cb2ae87cffc109a5b65d213e9df1",
    "build_date" : "2022-12-02T17:33:09.727072865Z",
    "build_snapshot" : false,
    "lucene_version" : "<luceneVersion>",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

If inPoint is not installed on the same server as Elasticsearch, both ports 9200 and 9300 TCP must be opened in the firewall.

Execute this command (as administrator)

netsh advfirewall firewall add rule name="ElasticSearch" protocol=TCP dir=in localport=9200,9300 action=allow profile=domain

Detailed Description see: Install Elasticsearch with .zip file
Elasticsearch download: Download Elasticsearch
This documentation is based on version: 7.17.15 elasticsearch-7.17.15-windows-x86_64.zip.

Set the JVM heap size

There are two possibilities to change the java memory settings (restart the ElasticSearch service afterwards):

Service manager

Open the service manager by using the following cmd: "bin\elasticsearch-service.bat manager ElasticSearch" (The last parameter is the %service id% - default is elasticsearch-service-x64)

This can be configured inside the service manager:

Registry

Open the registry HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\Apache Software Foundation\ProcRun 2.0\ElasticSearch\Parameters\Java

and set the registry values (all values in megabytes):

• JvmMs (Initial memory pool)
• JvmMx (Maximum memory pool)
• JvmSs (Thread stack size)

Upgrade from 7.17.8 to 7.17.15

IMPORTANT:

• Before you start read this article: Upgrade Elasticsearch
• Make sure, that no 5.x index is existing before installing (the setup will install, but ElasticSearch will not work). Migrate the index before the update!
• Make a full backup of the installation folder
• Start Kibana Upgrade assistent (Example: http://%COMPUTERNAME%:5601/app/management/stack/upgrade_assistant/overview) and make sure to fix the open steps
• Stop indexing (inPoint.Indexer service)
• Stop Elasticsearch service
• Rename the "jvm.options" file (located in the config folder of ElasticSearch) to "jvm.options_backup" so that the setup can install the newer version. After the setup succeeds migrate the required settings manually to the new file. The inPoint.Wizard will do this automatically (JVM heap size settings will be migrated).
• Open Programs and Features and uninstall the old ElasticSearch
• Extract the new ElasticSearch zip file to the new installation folder. (Example: C:\Program Files\Elastic\ElasticSearch\7.17.15) The new config folder (which includes the jvm.options and the elasticsearch.yml) is directly included in the zip file - please make sure to migrate the old settings (from previous step) to the new config files.
• Set the ES_PATH_CONFIG machine env variable to the config folder of ElasticSearch
• Set the ES_JAVA_HOME machine env variable to the current java installation directory (Example: C:\Program Files\Java\jdk-17.0.5+8)
• Set the ES_HOME machine env variable to the current ElasticSearch folder (Example: C:\Program Files\Elastic\ElasticSearch\7.17.15)
• Install ElasticSearch as a service (open CMD):
  • Go to the $ES_HOME$ folder
  • Execute: "\bin\ElasticSearch-Service.bat install ElasticSearch" (the last parameter is the %service id% - the inPoint.Wizard is using "ElasticSearch" - the default "elasticsearch-service-x64" will be used if you keep it empty)
  • Execute: net start ElasticSearch (inside CMD)
• Check if ElasticSearch is running

Upgrade from older versions

See previous release

Kibana

Kibana 7.17.15 Installation

Detailed Description see: Install Kibana on Windows
Kibana download: Download Kibana
This documentation is based on version: Kibana 7.17.15 as Windows (zip).

Extract the ZIP file to C:\\Program Files\\Kibana (as administrator)

To configure Kibana, the file kibana.yml must be opened. This file is located in the config folder of Kibana.
In this file the lines: server.host and elasticsearch.hosts have to be commented in and filled out correctly.

To install Kibana as a Windows service you have to download "NSSM": NSSM - the Non-Sucking Service Manager
Unzip the downloaded ZIP and browse to the folder win64.
Copy the file nssm.exe and paste it into C:\Program Files\Kibana.
Hold SHIFT key, right click on nssm.exe and select "Copy as path".
Start Command Prompt as Administrator and enter the following:
<the full path to nssm.exe (already in the clipboard)> install Elasticsearch-Kibana
Example: "C:\Program Files\Kibana\nssm.exe" install Elasticsearch-Kibana
This Window appears after enter was pressed:

Click on the "..." next to the "Path:" Line, then browse to the "Kibana/bin" folder and select kibana.bat file.
The "Startup directory" will be filled automatically:

Switch to the "Details" tab and enter the following:

• "Display name": Elasticsearch-Kibana
• "Description": Use Kibana to search, view, and interact with data stored in Elasticsearch indices.
• "Startup type": select "Automatic (Delayed Start)"

Switch to the "Dependencies" tab and enter the word "Elasticsearch".
Click on "Install service"

Installation is finished successful.

Open the Windows services and start "Elasticsearch-Kibana".

To quickly check if Kibana is running, open any Browser and call this URL: http://<host>:5601

If Kibana does not answer within 5 minutes, do the following:

1. Stop Elasticsearch-Kibana service
2. Open a Command Prompt (as Administrator) and navigate to C:\Program Files\Kibana\bin
3. Write kibana.bat and press enter
4. Wait until Server running at http://<host>:5601 is written
5. Refresh the Kibana Website in the Browser and make sure Kibana is shown correctly.
6. Press CTRL+C and then enter "y" in the Command Prompt
7. Start Elasticsearch-Kibana service

Port 5601 TCP must be opened in the firewall so that Kibana can also be reached externally.

Execute this command (as administrator)

netsh advfirewall firewall add rule name="ElasticSearch (Kibana)" protocol=TCP dir=in localport=5601 action=allow profile=domain

Secure Access to Kibana and ElasticSearch

If the Firewall rule was set without any IP restrictions, anyone who knows the Kibana URL, could access it and read/edit/delete any indices.

There are two ways to secure the communication:

1. Configure the firewall so that only the inPoint Server and the Elasticsearch Server are allowed to communicate with each other.
2. Set passwords in Elasticsearch.
   1. Enable security in elasticSearch.yml (after that, restart the ElasticSearch service (but don't start Kibana yet)

 xpack.security.enabled: true

2. Open a new Command Prompt (verify that the environment variable 'ES_PATH_CONF' is configured), navigate to C:\Program Files\Elastic\Elasticsearch\7.17.15\bin and then execute this commands:
   elasticsearch-setup-passwords.bat interactive
3. Then the password must be entered several times for different internal users.
4. Open the kibana.yml ( C:\Program Files\Kibana\config\kibana.yml ), remove comments to enable and fill out correctly the following lines:
   • elasticsearch.username (if required)
   • elasticsearch.password (if required)
5. After a Kibana restart you can continue.

Upgrade from 7.17.8 to 7.17.15

1. Stop Elasticsearch-Kibana service
2. Backup C:\Program Files\Kibana\config\kibana.yml configuration file
3. Rename the folder C:\Program Files\Kibana to Kibana_Backup
4. Extract downloaded Kibana ZIP File to C:\Program Files\Kibana
5. Open the kibana.yml ( C:\Program Files\Kibana\config\kibana.yml ), remove comments to enable and fill out correctly the following lines: - server.host - elasticsearch.hosts (value was named elasticsearch.url before) - elasticsearch.username (if required)
   - elasticsearch.password (if required)
6. Start Elasticsearch-Kibana service

Elasticsearch-Kibana service

If a different installation folder was chosen during the installation, the Elasticsearch-Kibana service must be adapted. Open a Command Prompt as Administrator, browse to the Folder where the nssm.exe exists and execute this command:

nssm.exe edit Elasticsearch-Kibana

Correct all paths in the opened Window.

Upgrade from older versions

See previous release

Indexer

.NET Framework 4.8 must be installed before the Indexer can be installed! Download .NET Framework 4.8

MSI started with a double click:
Select any language you like and then click "OK"

Click on "Next"

Click on "Next"

The Elasticsearch URL must be entered.
Example: http://<host>:9200
And if required, the user/password needed to connect to the Elasticsearch server.
Click on "Install"

If the installation was successful

The entered Elasticsearch URL, user name and password are written by the setup into the "inPoint.Indexer.exe.config" file.

inPoint.Indexer service was created:

The Indexer also checks our license, per default it was installed in C:\Program Files (x86)\HS Europe, copy the PAMLIC file into this folder.

Now the inPoint.Indexer service can be started.

If the indexer has successfully processed entries of the IP_FULLTEXT_QUEUE table, they are marked with STATUS = 2007.

• STATUS = 0
  The entries have not yet been processed by the indexer.
• STATUS = 100X
  The entries are currently being processed.
• STATUS = 2XXX
  The entries were processed.

The three digits after the 2XXX are a set from a flags enum. (see: inPoint.Fulltext.IndexStatus)

Here are a few that happen a lot:

• STATUS = 2007
  Item completely successfully indexed
• STATUS = 2006 Item (document) was indexed with all metadata from the DB, but no content of the file could be extracted
• STATUS = 2101 Item (folder) could no longer be found in the DB. None of this was indexed
• STATUS = 2102 Item (Share) could no longer be found in the DB. None of this was indexed

In later versions the STATUS = 2007 entries will be completely removed from the IP_FULLTEXT_QUEUE table.

NOTE: If a virus scanner is installed on the server, the temp directory used by the indexer must be excluded.

Backup and Restore

Backup

See: Elasticsearch Backing Up Your Cluster

First of all the path where the Backup should be created afterwards have to be defined. The Backup should be stored on a different hard disk than the one on which the index is written.

In this example, the Backup will be created at C:\Index\Backup.

Open the elasticsearch.yml and add this line:

path.repo: C:\\Index\_Backup

Save the elasticsearch.yml and restart Elasticsearch service

Open the Kibana "Dev Tool" Console and execute this command:

PUT _snapshot/inPoint_Backup
{
  "type": "fs",
  "settings": {
    "location": "C:\\Index_Backup"
  }
}

The Result should display:

{ "acknowledged": true }

Backup the index with this command:

PUT _snapshot/inPoint_Backup/bck_20181015
{
  "indices": "inpoint"
}

The Result should display:

{ "accepted": true}

It is also possible to specify all indices with "*" or several comma-separated.
The part "bck_20181015" can be chosen freely and should be uniquely defined for each backup operation so that the correct backup can be found in case of a restore.

Check the status of the Backup with this command:

GET _snapshot/inPoint_Backup/bck_20181015

Result:

Restore

See: Elasticsearch Snapshot and Restore

Close the currently used index:

POST inpoint/_close

Result:

{ "acknowledged": true }

Restore the Backup:

POST _snapshot/inPoint_Backup/bck_20181015/_restore

Result:

{ "accepted": true}

Open the index:

POST inpoint/_open

Result:

{ "acknowledged": true }

Cluster

Clustering can be used for performance reasons or as a safe guard against failure. (For more details: ElasticSearch Cluster)

NOTE:

• If you have two nodes the data duplication will be done automatically.
• At least 3 machines for resilience. (For more details: Availability cluster design)
• The new master node will be selected if you stop the current master node. The election is only working if you have more than half of the master eligible nodes up and running.
• If you need more nodes, then you have to add the ip adresses to the discovery.seed_hosts property inside ElasticSearch.yml on all nodes.

Configuration on a existing node (installed by the wizard):

• elasticsearch.yml
  • network.host
    Change to the fixed IP address of the node
  • discovery.seed_hosts - add the IP addresses or DNS names of all nodes
    e.g.:

    discovery.seed_hosts:
    -172.25.88.33
    -172.25.88.34
  • Remove discovery.type: single-node from Node1
• If necessary, open the ports in the firewall (default: 9200 and 9300)

Installation on a fresh node:

• Install OpenJDK 17 (64 Bit) JRE (see: OpenJDK 17 Installation)
• Install ElasticSearch (elasticsearch-7.17.15-windows-x86_64.zip (located in inPoint.Wizard files\ElasticSearch))
• After the installation - go to the ElasticSearch.yml and configure the following:
  • Set cluster.name (must be the same for all nodes, verify elasticsearch.yml on an already installed node)
  • Enter the IP address of the current machine in network.host
  • Add the IP addresses or DNS names of all existing and planned nodes to "discovery.seed_hosts:"
• Open the ports in the firewall (default: 9200 and 9300)
• Install Kibana (Its not needed for the cluster functionality)

Monitoring

All nodes which are currently running in a cluster can be monitored in ElasticSearch with the following URL http://<host>:9200/_cat/nodes?v=true or in Kibana under Monitoring.

ElasticSearch:

• Open http://<host>:9200/_cat/nodes?v=true

Kibana:

• Open http://<host>:5601/
• Click on Stack Management > Monitoring > Nodes