TxnStore Commands

TxnStore commands are a collection of commands used to work with TxnStore and its indices.


Following are the TxnStore commands:

HELP

The Help command displays the complete list of TxnStore commands along with their respective description.

Syntax Defined

Syntax: ae help txnstore

Alias Defined

Alias: ae help opensearch

Sample Output:

D:\RPA\AssistEdgeAutomation\scripts> ae help txnstore


Following options are available in AE TXNSTORE command.
 LISTROLES                           :        List all roles present is Txnstore 
 LISTUSERS                           :        List all local users for Txnstore 
 CREATEUSER                     :        Create a local user for Txnstore 
 DELETEUSER                      :        Delete Txnstore local user
 LISTREPO                             :        List repository.
 CREATEREPO                      :        Create repository in Txnstore .
 DELETEREPO                       :        Delete repository.
 BACKUP                               :        Take backup of Txnstore data.
 RESTORE                             :        Restore Txnstore data from backup.
 ACTION [ACTION FILE]      :        Execute custom action file using curator.
 HEAD [PORT]                      :        Starts Txnstore head plugin for debugging.

                                                   Any input required for action should be passed using env variable.

Txnstore internal users will be used for backup and restore.

For Backup and Restore, elasticsearch curator is used.
Please create Repository in Txnstore before performing backup or restore.
 https://opensearch.org/docs/1.2/opensearch/snapshot-restore
Provide Repository path in "ae.properties" file in key "TXNSTORE_REPO_PATH".
Restart Txnstore after updating "ae.properties" file.
In case of network shared location is used for backup and restore, Please
map it to local drive using "ae NETWROKDRIVE" command.

 

D:\RPA\AssistEdgeAutomation\scripts>

 

NOTE: 

For backward compatibility, ae elastic is still supported in all ae txnstore commands, for example: ae txnstore <Component Name> is executed as ae elastic <Component Name>.

 

LISTROLES

The listroles command displays a list of all available TxnStore roles.

Syntax Defined

Syntax: ae txnstore listroles

Alias Defined

Alias: ae opensearch listroles

Sample Output

D:\RPA\AssistEdgeAutomation\scripts>ae txnstore listroles


All roles are:
observability_read_access
snapshot_management_read_access
all_access
alerting_read_access
snapshot_user

 

D:\RPA\AssistEdgeAutomation\scripts>  

LISTUSERS

The listusers command displays the list of local TxnStore users and their roles.

Syntax Defined

Syntax: ae txnstore listusers

Alias Defined

Alias: ae opensearch listusers

Sample Output

D:\RPA\AssistEdgeAutomation\scripts>ae txnstore listusers


All roles are:

User                     Role
RPAUSER1           snapshot_user
RPAUSER2           notebooks_full_access 
RPAUSER3           asynchronous_search_read_access 

 

D:\RPA\AssistEdgeAutomation\scripts>

CREATEUSER

The createuser command creates a local TxnStore user. You must assign a role to this local user during creation.

Syntax Defined

Syntax: ae txnstore createuser

Alias Defined

Alias: ae opensearch createuser

Sample Output

D:\RPA\AssistEdgeAutomation\scripts>ae txnstore createuser


Please provide username : rpauser
Do you want to create it as a backup/snapshot user? (Y/N): N
Enter password (HIDDEN):
Enter role name: rpaitadmin
Creating user rpauser
user 'rpauser' created.


D:\RPA\AssistEdgeAutomation\scripts>


Enter details related to the user as mentioned below:

  • Please provide username: Provide the name of the user you want to create in the repository and then press <Enter>8.
  • Do you want to create it as a backup/snapshot user? (Y/N): If you want the user to be able to take backups/snapshots, enter Y; otherwise, enter N to provide the desired role and then press <Enter>8 .
  • Enter password (HIDDEN): Provide the password for the user and then press <Enter>8 .
  • Enter role name: Provide the role that you want to assign to the user and then press <Enter>8 .

NOTE:

There must be at least one user with snapshot_user role if you need to take TxnStore backup.

 

DELETEUSER

The deleteuser command is used to delete a local TxnStore user.

Syntax Defined

Syntax: ae txnstore deleteuser

Alias Defined

Alias: ae opensearch deleteuser

Sample Output

D:\RPA\AssistEdgeAutomation\scripts>ae txnstore deleteuser


Please provide username to be deleted: rpauser
Success: 'rpauser' deleted.


D:\RPA\AssistEdgeAutomation\scripts>

LISTREPO

The listrepo command displays the names and locations of all repositories created in TxnStore.

Syntax Defined

Syntax: ae txnstore listrepo

Alias Defined

Alias: ae opensearch listrepo

Sample Output

D:\RPA\AssistEdgeAutomation\scripts>ae txnstore listrepo


Name           Location
txnrepo        D:\RPA\AssistEdgeAutomation\data\Repo
reponew      D:\RPA\AssistEdgeAutomation\data\RepoNew

 

 D:\RPA\AssistEdgeAutomation\scripts> 

 

CREATEREPO

The createrepo command is used to create a repository for TxnStore. This repository is used to backup and restore the TxnStore indices.

Prerequisites

Before running the createrepo command, follow these steps: 

  • Stop all the AE components.
  • Create a new folder in the desired directory. For example - <Installation Directory>\AssistEdgeAutomation\data\Repo. Copy the path of new folder.
  • Open the ae.properties file in the <Installation Directory>\AssistEdgeAutomation\scripts folder and paste the folder path in TXNSTORE_REPO_PATH= line and save it. In case there are multiple repositories, the comma separated path can be provided.
  • Restart all the AE components.
  • In a cluster setup:
    • Network shared location should be used for creating the repository, for more information see Network Drive Commands.
    • ae.properties must be updated on all machines.
    • CREATEREPO command need to be executed only on one server.

NOTE: 

Two repositories should not point to the same path.

 

Syntax Defined

Syntax: ae txnstore createrepo

Alias Defined

Alias: ae opensearch createrepo

Sample Output

D:\RPA\AssistEdgeAutomation\scripts>ae txnstore createrepo
Please provide repository name: txnrepo
Please provide repository path:  D:\RPA\AssistEdgeAutomation\data\Repo
Repository Created.
D:\RPA\AssistEdgeAutomations\scripts>


Enter details related to the repository as mentioned below:

  • Please provide repository name: Provide the name of repository that you want to create in TxnStore and then press <Enter>8   .
  • Please provide repository path: Provide the repository path added in the ae.properties file and then press <Enter>8   .

DELETEREPO

The deleterepo command is used to permanently delete a repository from

TxnStore.

Syntax Defined

Syntax: ae txnstore deleterepo

Alias Defined

Alias: ae opensearch deleterepo

Sample Output

D:\RPA\AssistEdgeAutomation\scripts>ae txnstore deleterepo
Please provide repository name: txnrepo
Repository Deleted.
D:\RPA\AssistEdgeAutomation\scripts>

BACKUP

The backup command is used to take backup of TxnStore index/indices in an TxnStore repository.

Prerequisites

  • Make sure to create an TxnStore repository and add a user in TxnStore.
  • Ensure TxnStore is running before running the backup command.

Syntax Defined

Syntax: ae txnstore backup

Alias Defined

Alias: ae  opensearch backup

Sample Output

D:\RPA\AssistEdgeAutomation\scripts>ae txnstore backup
Please provide repository name: txnrepo
Please provide index name: *
Please provide Txnstore username: rpauser
Please provide Txnstore password (HIDDEN):
 2023-02-20 21:43:30,527 INFO      Preparing Action ID: backup, "snapshot"
2023-02-20 21:43:30,527 INFO      Creating client object and testing connection
2023-02-20 21:43:30,532 WARNING   Use of "http_auth" is deprecated. Please use "username" and "password" instead.
2023-02-20 21:43:30,532 INFO      Instantiating client object
2023-02-20 21:43:30,534 INFO      Testing client connectivity
2023-02-20 21:43:30,547 INFO      Successfully created Elasticsearch client object with provided settings
2023-02-20 21:43:30,554 INFO      Trying Action ID: backup, "snapshot":
2023-02-20 21:43:30,651 INFO      Creating snapshot "curator-20230220161330" from indices: ['security-auditlog-2023.02.20', 'rpa-apm-serversummary-2023.02.17', 'rpa-apm-server-2023.02.20', 'security-auditlog-2023.02.17']
2023-02-20 21:43:30,721 INFO      Snapshot curator-20230220161330 still in progress.
2023-02-20 21:43:40,729 INFO      Snapshot curator-20230220161330 successfully completed.
2023-02-20 21:43:40,737 INFO      Snapshot curator-20230220161330 successfully completed.
2023-02-20 21:43:40,737 INFO      Action ID: backup, "snapshot" completed.
2023-02-20 21:43:40,738 INFO      Job completed.
D:\RPA\AssistEdgeAutomation\scripts>


Enter details related to the repository as mentioned below:

  • Please provide repository name: Provide the repository name created in the TxnStore and then press <Enter>8   .
  • Please provide index name: Provide the index name for which you want to take backup, enter * to backup all indices and then press <Enter>8   .
  • Please provide Txnstore username: Provide the name of a local TxnStore user and has access to create TxnStore snapshot (snapshot_user role access) and then press <Enter>8   .
  • Please provide Txnstore password (HIDDEN): Provide the password of the user and then press <Enter>8   .

RESTORE

The restore command is used to restore TxnStore data from backup.

Prerequisites

  • Make sure you already have an TxnStore backup in a repository.
  • Ensure TXNSTORE is running before running the restore command.

Syntax Defined

Syntax: ae txnstore restore

Alias Defined

Alias: ae opensearch restore

Sample Output

D:\RPA\AssistEdgeAutomation\scripts>ae txnstore restore
Please provide repository name: txnrepo
Please provide index name: *
Please provide Txnstore username: rpauser
Please provide Txnstore password (HIDDEN):
2022-11-16 09:48:04,378 INFO      Preparing Action ID: close, "close"
2022-11-16 09:48:04,379 INFO      Creating client object and testing connection
2022-11-16 09:48:04,387 WARNING   Use of "http_auth" is deprecated. Please use "username" and "password" instead.
2022-11-16 09:48:04,388 INFO      Instantiating client object
2022-11-16 09:48:04,389 INFO      Testing client connectivity
2022-11-16 09:48:04,870 INFO      Successfully created Elasticsearch client object with provided settings
2022-11-16 09:48:04,878 INFO      Trying Action ID: close, "close":
2022-11-16 09:48:04,897 INFO          Closing 1 selected indices: ['security-auditlog-2022.11.16']
2022-11-16 09:48:04,947 INFO      Action ID: close, "close" completed.
2022-11-16 09:48:04,948 INFO      Job completed.
2022-11-16 09:48:05,745 INFO      Preparing Action ID: restore, "restore"
2022-11-16 09:48:05,746 INFO      Creating client object and testing connection
2022-11-16 09:48:05,755 WARNING   Use of "http_auth" is deprecated. Please use "username" and "password" instead.
2022-11-16 09:48:05,756 INFO      Instantiating client object
2022-11-16 09:48:05,759 INFO      Testing client connectivity
2022-11-16 09:48:05,781 INFO      Successfully created Elasticsearch client object with provided settings
2022-11-16 09:48:05,789 INFO      Trying Action ID: restore, "restore":
2022-11-16 09:48:05,798 INFO      Restoring indices "['security-auditlog-2022.11.16']" from snapshot: curator-20221116172046
2022-11-16 09:48:05,898 INFO      _recovery returned an empty response. Trying again.
2022-11-16 09:48:05,928 INFO      Provided indices: ['security-auditlog-2022.11.16']
2022-11-16 09:48:05,928 INFO      Found indices ['security-auditlog-2022.11.16']
2022-11-16 09:48:05,948 INFO      Found restored index security-auditlog-2022.11.16
2022-11-16 09:48:05,978 INFO      All indices appear to have been restored.
2022-11-16 09:48:05,978 INFO      Action ID: restore, "restore" completed.
2022-11-16 09:48:05,978 INFO      Job completed.
2022-11-16 09:48:06,587 INFO      Preparing Action ID: open, "open"
2022-11-16 09:48:06,587 INFO      Creating client object and testing connection
2022-11-16 09:48:06,596 WARNING   Use of "http_auth" is deprecated. Please use "username" and "password" instead.
2022-11-16 09:48:06,596 INFO      Instantiating client object
2022-11-16 09:48:06,598 INFO      Testing client connectivity
2022-11-16 09:48:06,621 INFO      Successfully created Elasticsearch client object with provided settings
2022-11-16 09:48:06,628 INFO      Trying Action ID: open, "open":
2022-11-16 09:48:06,650 INFO      Opening 1 selected indices: ['security-auditlog-2022.11.16']
2022-11-16 09:48:06,678 INFO      Action ID: open, "open" completed.
2022-11-16 09:48:06,678 INFO      Job completed.
D:\RPA\AssistEdgeAutomation\scripts>

 

Enter details related to the repository as mentioned below:

  • Please provide repository name: Provide the repository name created in the TxnStore where the backup of index/indices are already present and then press <Enter>8   .
  • Please provide index name: Provide the index name for which you want to take restore TxnStore data, enter * to restore all indices present in the backup and then press <Enter>8   .
  • Please provide Txnstore username: Provide the name of a user who exists in the TxnStore repository and then press <Enter>8   .
  • Please provide Txnstore password (HIDDEN): Provide the password of the user and then press <Enter>8   .

ACTION

The action command can be used if the user wants to perform additional operations on the opensearch data apart from the existing operations provided by AE. The actions that can be performed using the action command can be found at https://www.elastic.co/guide/en/elasticsearch/client/curator/current/index.html. 

Prerequisites

  • Create a custom .yml file.
  • Make sure to create an TxnStore repository and add a user in TxnStore.
  • Ensure TxnStore is running before running the backup command.

Syntax Defined

Syntax: ae txnstore action <location of action file>

Alias Defined

Alias: ae  opensearch action <location of action file> 

Sample Output

Following is an example of how to create the index by using an action file command:

 

D:\RPA\AssistEdgeAutomation\scripts>ae txnstore “ \AssistEdgeAutomation\ runtime\opensearch\plugins\curator-5.8.4\createindex.yml”
Please provide Txnstore username: Robert
Please provide Txnstore password (HIDDEN):
2023-02-20 22:22:55,006   INFO                         Preparing Action ID: 1, "create_index"
2023-02•20 22:22:55,009  INFO                         Creating client object and testing connection
2023-02-20 22:22:55,023 WARNING                  Use of "http_auth" is deprecated. Please use "username" and "password” instead.
2023-02-20 22:22:55,023   INFO                         Instantiating client object     
2023-02-20 22:22:55,025   INFO                         Testing client connectivity
2023-02-20 22:22:55,375   INFO                          Successfully created    Elasticsearch client object with provided settings
2023-02-20 22:22:55,381   INFO                          Trying Action ID: 1,    "create_index": Create the index as     named, with the specified extra settings.
2023-02-20 22:22:55,382   INFO                          Creating index “myindex” with settings: { ‘settings’: { ‘number_of_shards’ : 2, ‘number_of_replicas’ : 1}}
2023-02-20 22:22:56,009   INFO                          Action ID: 1, "create_index"  completed.
2023-02-20 22:22:56,010   INFO                          Job completed.
D:\RPA\AssistEdgeAutomation\scripts>


Enter details related to the repository as mentioned below:

  • Please provide Txnstore username: Provide the name of a user who exists in the TxnStore repository and then press <Enter>8   .
  • Please provide Txnstore password (HIDDEN): Provide the password of the user and then press <Enter>8   .

HEAD

The head command is used for debugging TxnStore. After running this command, a URL is generated; you can use this URL in a browser, along with local user details, to start the ElasticSearch head plugin for debugging.

Syntax Defined

Syntax:  txnstore <Head> <port>

Alias Defined

Alias: ae opensearch <Head> <port>

Sample Output

You must provide a port number that is not in use.
 

D:\RPA\AssistEdgeAutomation\scripts>ae txnstore head 8503
Please provide Auth information in URI.
http://localhost:8503/?auth_user=USERNAME&auth_password=PASSWORD
ElasticSearch Head started at port: 8503