hologos/tpcp

A smart tool to remotely work with SAP import queue and transports.

tpcp's Issues

Check the code against ShellCheck

Process a transport for all systems from landscape without the need to specify all systems

It would be nice to process a transport for all systems from landscape specifying only system group.

Ability to set TPCP_LOG_DIRPATH_ROOT from outside

I need to set the directory to store all logs from outside.

Caching to SAPTRANSPORT_TRANSPORT_DOMAINS_MAP is broken

Since processing actions is run as background job, filling data to SAPTRANSPORT_TRANSPORT_DOMAINS_MAP array in one thread doesn't affect the array in other threads.

Load system information from a configuration file

System informations are now hardcoded in the script. It needs to be loaded from a configuration file.

Log messages are printed to STDERR and needs to be redirected to a file

Log messages are printed to STDERR and needs to be redirected to a file with 2> filename or it breaks the rendering.

SSH multiplexing doesn't work if multiple thread tries to connect to the same host

Needs futher investigation. Here is what I think is happening:

If multiple threads tries to connect to the same host, the very first one detects that no master connection is created. So it creates one. But in meantime other threads detect that master connection is opened but it actually isn't fully initialized yet.

Skip COPY action on non-domain controller systems if multiple system from same transport domain are present

Cleaning of temp files (workdir) is missing in traps

I removed calling of tpcp::utils::cleanup() in traps but don't remember the exact reason. It had something to do with my shell (zsh) automatically calling traps even on successful execution of the script. Need more investigating.

Caching transport domain on single system with 2 different clients

Caching domain controller SID on single system with 2 different clients overwrites identical tmp file.

tpcp AAAK000001 ZZZ000 ZZZ999 ADD

Provide bash completion

Maybe implement associative arrays?

Go through the code, maybe associative arrays would simplifies things.

Save STDERR output per transport+system to a file

It's hard for end-user to find what caused an error.

Add more log messages for end-user

The end user knows only about errors.

Killing main program doesn't kill background jobs (threads)

Sending SIGING kills only main program, background jobs keeps running.

Load domain controller SID from /usr/sap/trans/bin/DOMAIN.CFG

There is a IF statement now in the script, load it from /usr/sap/trans/bin/DOMAIN.CFG.

Import of transport rq end with error, stop processing next requests

When I try to import more then one transports. And some import failed, tpcp continue with import next transport requests. Is better end processing after first import error.

Fix array usage without "

Looping over an array not declared with " can split the element into multiple values. The same goes for passing array as argument.

Import ended (4) Ended with Warning: it's okay

When import of transport rq ends with RC= 4 -> It's okay.
But tpcp ends with error.

Domain controller gets cached twice

Right now, if domain controller is cached, it gets cached twice:

[23.11.2019 14:11:57] D: AED: Caching transport domain for AED.
[23.11.2019 14:11:58] D: AED: Transport domain for AED is AED.
[23.11.2019 14:11:58] D: AED: Caching transport domain for domain controller AED.

Loading of system informations is very slow

Function saptransport::load_ini_file is very slow

time saptransport::load_ini_file "${TPCP_CONFIG_FILEPATH}" 1>&2

real    0m2,369s
user    0m1,051s
sys     0m1,301s

Suggest a command to retry what failed

If something fails (for whatever reason), it would be great if the program would suggest a command sequence to retry what failed.

If there is a problem with input parameters, user doesn't know, what happened

Since the output is redirected to a file, user doesn't know, what happened. The program just exists without any explanation.

The output gets overwritten if it doesn't fit on the screen

The output gets overwritten because the cursor cannot get to negative positions (it cannot scroll).

Also if 2 transports are processed in a single run, the second overwritten the first one.

Initialize all variables used in for loop.

Not all variables are initialized for use in for loop. Bash doesn't initialize variable used in for loop definition and it can cause some serious bugs.

local lang=""

for lang in "${langs[@]}"; do
    ....
done

vs

for lang in "${langs[@]}"; do
    ....
done

Do not allow duplicate values in input

It can cause some serious misbehaviour of the program.

Implement tp SUBSET

Support for tp SUBSET is needed.

Owner of tranport files isn't always set to domain controller

Copying transport files to non-domain controller system make the owner as non-domain controller.

The script continues to execute if caching domain controllers fails

Since caching domain controllers is run in the background, failing it doesn't terminate the program.

Generate a csv file for tracking what has been done

To track what has happened, generate a csv file.

Error messages show on the screen

Since the invocation of the script is not done with 2> debug.log, error messages show on the screen.

Thread in WAITING status can cause infinite loop

If a thread that is copying files dies and doesn't change its status, all threads that are waiting for it to finish will never break from the waiting loop.

Redraw rendered screen faster

Since bash-4.4 is required, it support sleep with fractions. Use it to make the program more responsive.

Rename transport domain to domain controller

In reality domain controllers are cached, not transport domain.

Enhance tpcp::utils::usage() with formatting

Add formatting to tpcp::utils::usage() using fidian/ansi.

Parse input parameters separately instead of as whole list

It is really annoying to separate parameters with " or '. Since the format of transport, system and actions are very distinc, parse it according to format.

Don't copy transport to multiple systems in the same transport domain

Copying transport to multiple systems in the same transport domain leads to errors and to files being overwritten.

Update README.md

A lot of what is written is not true anymore.

Don't copy files to target system if transport source is in the same transport domain

If I want to copy transport files to multiple systems (and some are in the same transport domain), don't copy the files. They would be overwritten and it could corrupt them.

For example, I want to do everything with a single command:

• add, import and delete transport to AA* systems
• copy, add, import and delete transport to BB* systems

# AA0, AA1, AA2 are in the same transport domain
tpcp AA0K999999 AA1000 AA2000 BB1000 BB2000 CPY ADD IMP DEL

                          CPY ADD IMP DEL
AA0K999999     AA1000      -   ✔   ✔   ✔
               AA2000      -   ✔   ✔   ✔
               BB1000      ✔   ✔   ✔   ✔
               BB2000      ✔   ✔   ✔   ✔

Commenting out logger::log speeds things significantly

Commenting calls of logger::log() in saptransport::load_ini_file() speeds things up (from ~1000 ms to ~50 ms).

SSH timeout doesn't work when hostname cannot be resolved

Even though -o ConnectTimeout=5 is set with ssh, it takes ages to timeout if hostname cannot be resolved.

Terminating action threads without changing their status can cause infinite loop

Every function called from within saptransport::__run_action_on_system() cannot terminate unless it changes the status. Make them return 1 and deal with in in saptransport::__run_action_on_system().

Test the program under bash v4.4

I found in bash v5.0 release notes the following:

1. New Features in Bash
   ...
   m. If a non-interactive shell with job control enabled detects that a foreground
   job died due to SIGINT, it acts as if it received the SIGINT.

It might mean that my recent fix for killing background jobs work only in bash v5.0, which I tested it on.

Manage dependencies with peru

Input parser doesn't recognize missing parameters

$ TPCP_LOGGER_LEVEL=D ./tpcp AADK000001
[25.11.2019 15:28:08] I: Creating log directory './tpcp-logs/debug'.

Loading system informations from /Users/malekjir/.config/tpcp/system-definition.ini.

Caching information about transport domains.
./libs/saptransport: line 24: !2: unbound variable
./libs/saptransport: line 366: /tmp/tpcp.20191125-152808.9489.26601/saptransport_status.*: No such file or directory



Logs are located at ./tpcp-logs/debug.

Build using TravisCI

Build releases using TravisCI.

Remove sg_name and sg_role

sg_name and sg_role are not being used, no reason for keeping them.

Use /dev/shm on linux instead of /tmp

Filesystem /dev/shm is a ram disk (much better IOps).

Print info about transport progress to main.log

Currently the main.log doesn't contain information about progress of transports with default logger level. I want to see at glance, what transports and to what systems were those processed and their final status.

DEL ends with error if transport is not in an import queue

To be able to add all transports in an order, I have to first run DEL action, then ADD action. If some transport is not in an import queue, it then ends with an error. Running DEL on transport not in an import queue should be "soft" error.