Version: 2024.3.1

inPoint.CloudSync

inPoint.CloudSync is a service to synchronize documents saved in Microsoft Office 365 or Microsoft Teams with a counterpart archived in inPoint. The basic configuration set up the general connection to Microsoft and how to connect to inPoint. The administration then configures which folders are synchronized or imported.

Basic Configuration​

For installing the msi have a look at the installation documentation.

The settings are defined in the file appsettings.json. Most of these settings are automatically filled by the setup.
Default: "C:\Program Files (x86)\HS Europe\inPoint.CloudSync\appsettings.json"

Full Sample​

{
    "O365": {
        "Instance": "[O365_INSTANCE]",
        "Tenant": "[O365_TENANT]",
        "ClientId": "[O365_CLIENTID]",
        "ClientSecret": "[O365_CLIENTSECRET]",
        "ClientCertificate": "[O365_CLIENTCERT]",
        "NotificationUrl": "[O365_NOTIFICATIONURL]"
    },
    //"Urls": {
    //    "inPointCloudSync": "[CLOUDSYNCURL]",//default is to read from central registry (override only when needed)
    //    "inPointIdentity": "[IDENTITYURL]",  //default is to read from central registry (override only when needed)
    //    "inPointCore": "[COREURL]"           //default is to read from central registry (override only when needed)
    //},

    "SyncSecurity": true,
    "SecurityFolderLevel": 100,
    "CleanUpEmptyFolderLastModified": "00:30",
    "CleanUpEmptyFolderInterval"    : "01:00",

    "Host": {
        "Certificates": {
            "Default": {
                "Path": "[CERTPATH]",
                "Password": "[CERTPASSWORD]",
                "Subject": "[CERTSUBJECT]",
                "Store": "[CERTSTORE]",
                "Location": "[CERTLOCATION]"
            }
        }
    },
    "Serilog": {
        // https://github.com/serilog/serilog/wiki/Configuration-Basics
        "MinimumLevel": {
            "Default": "Information",
            "Override": {
                "Microsoft": "Warning",
                "System": "Warning"
            }
        },
        "Enrich": [ "FromLogContext", "WithMachineName", "WithThreadId" ],
        "WriteTo": [
            {
                // https://github.com/serilog/serilog-sinks-file
                "Name": "File",
                "Args": {
                    "path": "c:\\temp\\inPoint.CloudSync_.log",
                    "fileSizeLimitBytes": 1000000000,
                    "retainedFileCountLimit": 21,
                    "rollingInterval": "Day",
                    "buffered": true,
                    "outputTemplate": "[{Timestamp:HH:mm:ss} {Level:u3}] {Message}{NewLine}{Exception}"
                }
            }
        ]
    }
}

O365 - Connect to Office 365​

This section defines how the service will connect to Microsoft Azure AD (Microsoft O365 or Microsoft Teams). See the Connect to Office 365 in the installation documentation.

Instance​

The instance of Azure AD. Default: https://login.microsoftonline.com/{0}

Tenant​

The tenant in Azure AD. This is a guid similar to this sample 5529cdf5-e60e-4945-9ad0-e13a5a6be8cb. It' visible in the Application Details in Azure AD as well.

ClientId​

The application ID of inPoint.CloudSync from registering it in Azure AD. It' visible in the Application Details in Azure AD as well.

ClientSecret​

The client secret or password generated in Azure AD. If using ClientSecret, the value for ClientCertificate must be empty!

ClientCertificate​

The client certificated in Azure AD, which has to be generated or installed on the server first. If using ClientCertificate, the value for ClientSecret must be empty!

NotificationUrl​

If the server itself is not reachable from the internet, it's required to use some kind of proxy or forward to allow notifications from Microsoft to reach inPoint.CloudSync. The configured URL must forward all request to the installed service with the path /api/notification at the end. The notification callback can be verified using this command.

Urls - Connect to inPoint​

This section defines how the service exposes itself and how it will connect to inPoint. See the Connect to inPoint in the installation documentation.

All the Urls can be empty, because they can be read from the registry.
HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\HS Europe\Config\Urls

inPointCloudSync​

This is the URL which the service will be using for itself.

inPointIdentity​

The url to inPoint.Identity.

inPointCore​

The url to inPoint.Core.

SyncSecurity​

Default: false

If set to true, the security from Azure will be imported into inPoint. If false all new folder will always inherit the security from their parent (except for the first folder levels defined by SecurityFolderLevel which only allows the site owners).

Requirements
The users and groups must already exist in inPoint and have a mail address! All other users and groups are ignored.

SecurityFolderLevel​

Default: 2

The meaning of this value depends on the value of SyncSecurity.

"SyncSecurity": true​

The number of levels which have the security taken from Azure. Use -1 to import it for all levels. The deeper levels use inherited permissions.

"SyncSecurity": false​

The number of levels which use the inPoint site owner security when creating new folders. The deeper levels will use inherited permissions. This ensures that new folders (e.g. a Teams folder) are only visible to an administrator who has to change the permission afterwards to allow access to the correct users. After this change all the sub-folders and all new created sub-folders will be visible to those users as well.

CleanUpEmptyFolderLastModified​

Default: "00:30"
(the format is hh:mm or hh:mm:ss, so the default value means 30 minutes)

If empty folder are deleted from Azure, this is the minimum age they must have before they are deleted. The folder cleanup has to be enabled on the subscription level.

CleanUpEmptyFolderInterval​

Default: "01:00"
(the format is hh:mm or hh:mm:ss, so the default value means 60 minutes)

If empty folder are deleted from Azure, this is the interval to search for empty folders.

Host - SSL-Certificate​

This section configures the ssl-certificate which is used for hosting the service. It can be configured using an certificate in the file-system or in the list of the user or computer certificates. See how to Configure the web-service in the installation documentation. For more details about certificates read here!

Serilog - Logging​

inPoint.CloudSync uses a comprehensive structured logging system. The default is to log only warnings and errors to 'c:\temp\inPoint.CloudSync_YYYMMDD.log' (where 'YYYMMDD' is the current date).

For more details look at Serilog and the File sink

Logging levels​

The MinimumLevel configures the logging levels for all inPoint.Core systems;

  • Default are the inPoint.Core application logs
  • Microsoft are the internal Microsoft ASP.NET logs
  • System are the internal Microsoft .NET logs

Usually changing the Default level (application logging) should be enough.
Available logging levels:

LevelUsage
VerboseVerbose is the noisiest level, rarely (if ever) enabled for a production app.
DebugDebug is used for internal system events that are not necessarily observable from the outside, but useful when determining how something happened.
InformationInformation events describe things happening in the system that correspond to its responsibilities and functions. Generally these are the observable actions the system can perform.
WarningWhen service is degraded, endangered, or may be behaving outside of its expected parameters, Warning level events are used.
ErrorWhen functionality is unavailable or expectations broken, an Error event is used.
FatalThe most critical level, Fatal events demand immediate attention.

Setting the logging level​

When you need to analyse an issue you should:

  • raise the MinimulLevel/Default from Warning to Debug
  • change the WriteTo/RollingFile/buffered to false in order to immediately save to the log file (otherwise it's buffered and you'll have to stop the Service to see the latest logs)

NOTE: do not forget to restore to their original values as these settings will negatively impact performance!

Administration​

The configuration is done via command line!

Open a command line in the installation directory and execute the following statement to get the help!

inPoint.CloudSync.exe /Config /?

A cloud is the connection between the Microsoft and the inPoint world. A single inPoint.CloudSync service will only process the clouds which match the tenant from the configuration file. Each cloud can have multiple subscription which configures the folder on both sides and how the synchronization or import should be done.

Show the configuration​

There are two commands to show the current configuration.

To show the configuration from all clouds which will be processed by this instance use one of these two:

inPoint.CloudSync.exe /Config
inPoint.CloudSync.exe /Config /List

To show the configuration from all clouds configured in the database use:

inPoint.CloudSync.exe /Config /ListAll

Cloud​

Adding a Cloud​

Creates a cloud for the current instance or optional for another instance. After executing the command the Id of the new cloud is returned.

inPoint.CloudSync.exe /Config /AddCloud  </other parameter...>
Name*​

<text>
The name of this cloud, all chars are allowed but it is recommended to use only chars which are also allowed in filenames.

inPointTenant*​

<number>
The id of the inPoint tenant

ArchiveId*​

<number>
The id of the inPoint archive

Type​

O365|NotSet
The type of the cloud. Currently only O365 can be processed and other values are ignored by the service.

TenantCloud​

<guid>
The tenant of the cloud provider (if omitted the value 'O365.Tenant' from appsettings.json is used! A different value means that this instance will not process this cloud).

noCheck​

Whenever a cloud is created or modified some checks are done (e.g. if the inPointTenant is valid). With this switch the checks can be turned off.

Editing a Cloud​

Edits an existing cloud given by its id. For other parameter see /AddCloud.

inPoint.CloudSync.exe /Config /EditCloud <id>  </other parameter...>

Deleting a Cloud​

Deletes a cloud and optionally deletes all subscription too.

inPoint.CloudSync.exe /Config /DeleteCloud <id>  [/force]
force​

Forces a delete even if subscriptions are existing.

Subscription​

Adding a Subscription​

Adds a subscription to an existing cloud.

inPoint.CloudSync.exe /Config /AddSub[scription]  </other parameter...>
*Resource​

<path>
Can be one of these two types:

  • The path to a resource (starting with '/', e.g. /groups/556ef2dd-0a9f-4f0f-9bf8-3d0bcb768193/drive/root
  • The web path to a O365/SharePoint location (starting with 'https:')
FolderUri​

<itemUri>
The itemUri of the inPoint target folder (must be set when deleting files from O365/SharePoint.

DeleteOnArchive​

false|true
If true the files are deleted from the O365/SharePoint location after archiving into inPoint. (empty folders are deleted as well, the interval and the age are configured once per instance.)

Excludes​

<regex>
A regular expression to exclude files. The default is ?~.*.tmp$ use an empty value to include everything

ClientState​

<text>
When requesting a change notification this text will be used to verify that the notification is originated from the server where it was requested.

Delta​

<text>
The last delta (the id of the last change). When this is reseted to empty a full synchronization will be done.

CloudId​

<int>
The cloud to which to add the subscription. If omitted the cloud with the same 'O365.Tenant' from appsettings.json is used.

RetentionMonths​

<int>
An optional number of months to protect against deletion (use -1 to protect forever and 0 for no protection).

Extra​

<json> An optional json formatted list of fixed values to include in the creation/update of documents. (remember to escape the double-quotes with two double-quotes on the command line)

noCheck​

Whenever a subscription is created or modified some checks are done (e.g. if the inPoint FolderUri is valid). With this switch the checks can be turned off.

Editing a Subscription​

Edits an existing subscription given by its id. For other parameter see /AddSub.

inPoint.CloudSync.exe /Config /EditSub[scription] <id> </other parameter...>

Deleting a Subscription​

Deletes a subscription and optionally deletes all items too.

inPoint.CloudSync.exe /Config /DeleteSub[scription] <id> [/CloudId <id>] [/force]
force​

Forces a delete even if items are existing

Troubleshooting​

Run a background task manually​

Common Parameter​

These parameters are accepted by all commands.

CloudId​

<int>
The cloud to synchronization. Can be empty if only one cloud is configured for the tenant.

SubscriptionId​

<list of int>
A comma separated list of subscription ids to sync (can be empty to synchronize all).

Manually running a synchronization​

Run a synchronization (using the delta from the last run):

inPoint.CloudSync.exe /SyncNow /CloudId 1 /SubscriptionId 1,2

Manually trigger a cleanup of empty folders​

Manually trigger a cleanup of empty folders of a subscription. Only subscriptions with DeleteOnArchive set to true will be processed.

inPoint.CloudSync.exe /ClearEmptyFolder /CloudId 5 /SubscriptionId 1

List files (all or changed)​

Save the list of files and folders from one or more subscription to a CSV file.

Output​

Writes the following columns separated by a semicolon to the file or the console (if no file is given):

  • CloudId (the id of the cloud)
  • SubId (the id of the subscription)
  • ItemId (the id of the item from SharePoint)
  • WebUrl (the full URL of the item in SharePoint)
  • CreationDate (the creation date of the item in UTC)
  • LastModifiedDate (the last modification date of the item in UTC)
  • FileType (the mime-type of the item, will be empty for folders)
  • Size (the size of the item)

Parameters​

Allows the same common parameters as the synchronization.

Log​

<file path>
If set the output is written to this file.

delta​

<id>
For fetching all files omit this parameter, use a specific value or use db to read the delta from the database. In case the delta is too old, a full fetch is done in the background (similar to a normal synchronization).

Example​

List all files since the last run by using the delta from the database

inPoint.CloudSync.exe /dump /SubscriptionId 8 /log items.csv /delta db

Verify the notification callback​

This will execute the notification callback and return the result. For a correct test this should be executed from the outside of the company network. (testing with a web browser is not the same, because the browser will use a "GET" instead of a "POST")

Use the correct NotificationUrl from the configuration file, but keep the parameter after the question mark.

Invoke-WebRequest -Method Post -Uri 'https://server.com:5001/api/notification?validationToken=myToken'

This is the expected result, the status code must be 200 and the content must be the same as the value of 'validationToken'.

StatusCode        : 200
StatusDescription : OK
Content           : myToken
RawContent        : HTTP/1.1 200 OK
                    Content-Length: 7
                    Content-Type: text/plain; charset=utf-8
                    Date: Wed, 06 Dec 2023 11:03:34 GMT
                    Server: Apache
                    
                    myToken
Headers           : {[Content-Length, 7], [Content-Type, text/plain; charset=utf-8], [Date, Wed, 06 Dec 2023 11:03:34 GMT], [Server, Apache]}
RawContentLength  : 7