Giter Site home page Giter Site logo

config's Introduction

config

my config and TIL stuff in issues

config's People

Contributors

jackblk avatar

Watchers

 avatar

config's Issues

wget download all files recursively

wget -rcv -np -nv -e robots=off --reject-regex=thumb https://example.com/wp-content/uploads/photo-gallery/...

another script:

#!/bin/bash
echo "Enter URL: "
read url
echo " "
echo "Enter file type: "
read type
wget -r -A.$type $url
mkdir -p recovered.files
cp -v -r \find . -name "*.$type" ./recovered.files
exit

Setting up new user in Linux

The best way:

sudo useradd -m -s /bin/bash newuser
# -m: create home directory
# -s: set shell

# set password
sudo passwd newuser

# add to sudo group
sudo usermod -aG sudo newuser
sudo usermod -aG docker newuser

Change default shell: https://www.baeldung.com/linux/change-default-shell

# add new user
sudo useradd USERNAME

# change default shell
sudo chsh -s /bin/bash USERNAME

# login user
sudo su USERNAME

Feeling lucky

{google:baseURL}search?q=%s&btnI&sourceid=navclient&gfns=1

or

https://google.com/search?q=%s&btnI&sourceid=navclient&gfns=1

Oracle Cloud mount Paravirtualized Disk

Check the connected volume

# check if the disk is connected
ls -l /dev/oracleoci/oraclevd*

# make file system for a new disk
sudo mkfs -t ext4 /dev/oracleoci/oraclevdb

# mount to a new folder
sudo mkdir /datadisk
sudo mount /dev/oracleoci/oraclevdb /datadisk

# check disk free
df -h

Adding the volume to fstab

# check uuid of block vol
blkid

# add it to /etc/fstab
vi /etc/fstab

New line of fstab should be:

UUID="925f47c8-5756-4d72-8120-028a9316c94d" /datadisk ext4 defaults,noatime,_netdev,nofail 0 2

Ref:

Disable SSH password and use publickey only

sudo vi /etc/ssh/sshd_config

Set these values:

ChallengeResponseAuthentication no
PasswordAuthentication no
UsePAM no

# disable root login
PermitRootLogin no
PermitRootLogin prohibit-password

Reload ssh:

sudo systemctl reload ssh

reference

Fix innodb incorrect file format corrupt in mariadb/mysql

Using https://hub.docker.com/r/bitnami/mariadb, I'm having this issue, can't start the container at all.

Checking logs with env var BITNAMI_DEBUG=1:

$ docker logs -f mariadb 
mariadb 11:43:34.29 
mariadb 11:43:34.30 Welcome to the Bitnami mariadb container
mariadb 11:43:34.30 Subscribe to project updates by watching https://github.com/bitnami/containers
mariadb 11:43:34.31 Submit issues and feature requests at https://github.com/bitnami/containers/issues
mariadb 11:43:34.31 
mariadb 11:43:34.32 INFO  ==> ** Starting MariaDB setup **
mariadb 11:43:34.38 INFO  ==> Validating settings in MYSQL_*/MARIADB_* env vars
mariadb 11:43:34.41 INFO  ==> Initializing mariadb database
mariadb 11:43:34.42 DEBUG ==> Ensuring expected directories/files exist
mariadb 11:43:34.44 INFO  ==> Updating 'my.cnf' with custom configuration
mariadb 11:43:34.45 INFO  ==> Setting user option
mariadb 11:43:34.46 DEBUG ==> Setting user to 'bn_wordpress' in mariadb configuration file /opt/bitnami/mariadb/conf/my.cnf
mariadb 11:43:34.49 INFO  ==> Setting slow_query_log option
mariadb 11:43:34.49 DEBUG ==> Setting slow_query_log to '0' in mariadb configuration file /opt/bitnami/mariadb/conf/my.cnf
mariadb 11:43:34.50 INFO  ==> Setting long_query_time option
mariadb 11:43:34.50 DEBUG ==> Setting long_query_time to '10.0' in mariadb configuration file /opt/bitnami/mariadb/conf/my.cnf
mariadb 11:43:34.51 INFO  ==> Using persisted data
mariadb 11:43:34.57 INFO  ==> Running mysql_upgrade
mariadb 11:43:34.58 INFO  ==> Starting mariadb in background
2022-12-25 11:43:34 0 [Note] /opt/bitnami/mariadb/sbin/mysqld (server 10.9.4-MariaDB) starting as process 72 ...
2022-12-25 11:43:34 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
2022-12-25 11:43:34 0 [Note] InnoDB: Using transactional memory
2022-12-25 11:43:34 0 [Note] InnoDB: Number of transaction pools: 1
2022-12-25 11:43:34 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions
2022-12-25 11:43:34 0 [Note] mysqld: O_TMPFILE is not supported on /opt/bitnami/mariadb/tmp (disabling future attempts)
2022-12-25 11:43:34 0 [Note] InnoDB: Using Linux native AIO
2022-12-25 11:43:34 0 [Note] InnoDB: Initializing buffer pool, total size = 128.000MiB, chunk size = 2.000MiB
2022-12-25 11:43:34 0 [Note] InnoDB: Completed initialization of buffer pool
2022-12-25 11:43:34 0 [Note] InnoDB: File system buffers for log disabled (block size=4096 bytes)
2022-12-25 11:43:34 0 [Note] InnoDB: 128 rollback segments are active.
2022-12-25 11:43:34 0 [Note] InnoDB: Setting file './ibtmp1' size to 12.000MiB. Physically writing the file full; Please wait ...
2022-12-25 11:43:34 0 [Note] InnoDB: File './ibtmp1' size is now 12.000MiB.
2022-12-25 11:43:34 0 [Note] InnoDB: log sequence number 47867740; transaction id 28273
2022-12-25 11:43:34 0 [Note] InnoDB: Loading buffer pool(s) from /bitnami/mariadb/data/ib_buffer_pool
2022-12-25 11:43:34 0 [Note] Plugin 'FEEDBACK' is disabled.
2022-12-25 11:43:34 0 [Note] Server socket created on IP: '127.0.0.1'.
2022-12-25 11:43:34 0 [Note] InnoDB: Buffer pool(s) load completed at 221225 11:43:34
2022-12-25 11:43:34 0 [Warning] 'proxies_priv' entry '@% root@a95d405e1aee' ignored in --skip-name-resolve mode.
2022-12-25 11:43:34 0 [Note] /opt/bitnami/mariadb/sbin/mysqld: ready for connections.
Version: '10.9.4-MariaDB'  socket: '/opt/bitnami/mariadb/tmp/mysql.sock'  port: 3306  Source distribution
Phase 1/7: Checking and upgrading mysql database
Processing databases
mysql
mysql.column_stats
Error    : Incorrect file format 'column_stats'
error    : Corrupt
mysql.columns_priv                                 OK
mysql.db                                           OK
mysql.event                                        OK
mysql.func                                         OK
mysql.global_priv                                  OK
mysql.gtid_slave_pos                               OK
mysql.help_category                                OK
mysql.help_keyword
Error    : Incorrect file format 'help_keyword'
error    : Corrupt
mysql.help_relation
Error    : Incorrect file format 'help_relation'
error    : Corrupt
mysql.help_topic
Error    : Incorrect file format 'help_topic'
error    : Corrupt
mysql.index_stats
Error    : Incorrect file format 'index_stats'
error    : Corrupt
mysql.innodb_index_stats                           OK
mysql.innodb_table_stats                           OK
mysql.plugin                                       OK
mysql.proc                                         OK
mysql.procs_priv                                   OK
mysql.proxies_priv                                 OK
mysql.roles_mapping                                OK
mysql.servers                                      OK
mysql.table_stats
Error    : Incorrect file format 'table_stats'
error    : Corrupt
mysql.tables_priv                                  OK
mysql.time_zone                                    OK
mysql.time_zone_leap_second                        OK
mysql.time_zone_name                               OK
mysql.time_zone_transition                         OK
mysql.time_zone_transition_type                    OK
mysql.transaction_registry                         OK

Repairing tables
mysql.column_stats
Error    : Incorrect file format 'column_stats'
error    : Corrupt
mysql.help_keyword
Error    : Incorrect file format 'help_keyword'
error    : Corrupt
mysql.help_relation
Error    : Incorrect file format 'help_relation'
error    : Corrupt
mysql.help_topic
Error    : Incorrect file format 'help_topic'
error    : Corrupt
mysql.index_stats
Error    : Incorrect file format 'index_stats'
error    : Corrupt
mysql.table_stats
Error    : Incorrect file format 'table_stats'
error    : Corrupt
Phase 2/7: Installing used storage engines... Skipped
Phase 3/7: Fixing views
mysql.user                                         OK
sys.host_summary                                   OK
sys.host_summary_by_file_io                        OK
sys.host_summary_by_file_io_type                   OK
sys.host_summary_by_stages                         OK
sys.host_summary_by_statement_latency              OK
sys.host_summary_by_statement_type                 OK
sys.innodb_buffer_stats_by_schema                  OK
sys.innodb_buffer_stats_by_table                   OK
sys.innodb_lock_waits                              OK
sys.io_by_thread_by_latency                        OK
sys.io_global_by_file_by_bytes                     OK
sys.io_global_by_file_by_latency                   OK
sys.io_global_by_wait_by_bytes                     OK
sys.io_global_by_wait_by_latency                   OK
sys.latest_file_io                                 OK
sys.memory_by_host_by_current_bytes                OK
sys.memory_by_thread_by_current_bytes              OK
sys.memory_by_user_by_current_bytes                OK
sys.memory_global_by_current_bytes                 OK
sys.memory_global_total                            OK
sys.metrics                                        OK
sys.processlist                                    OK
sys.ps_check_lost_instrumentation                  OK
sys.schema_auto_increment_columns                  OK
sys.schema_index_statistics                        OK
sys.schema_object_overview                         OK
sys.schema_redundant_indexes                       OK
sys.schema_table_lock_waits                        OK
sys.schema_table_statistics                        OK
sys.schema_table_statistics_with_buffer            OK
sys.schema_tables_with_full_table_scans            OK
sys.schema_unused_indexes                          OK
sys.session                                        OK
sys.session_ssl_status                             OK
sys.statement_analysis                             OK
sys.statements_with_errors_or_warnings             OK
sys.statements_with_full_table_scans               OK
sys.statements_with_runtimes_in_95th_percentile    OK
sys.statements_with_sorting                        OK
sys.statements_with_temp_tables                    OK
sys.user_summary                                   OK
sys.user_summary_by_file_io                        OK
sys.user_summary_by_file_io_type                   OK
sys.user_summary_by_stages                         OK
sys.user_summary_by_statement_latency              OK
sys.user_summary_by_statement_type                 OK
sys.version                                        OK
sys.wait_classes_global_by_avg_latency             OK
sys.wait_classes_global_by_latency                 OK
sys.waits_by_host_by_latency                       OK
sys.waits_by_user_by_latency                       OK
sys.waits_global_by_latency                        OK
sys.x$host_summary                                 OK
sys.x$host_summary_by_file_io                      OK
sys.x$host_summary_by_file_io_type                 OK
sys.x$host_summary_by_stages                       OK
sys.x$host_summary_by_statement_latency            OK
sys.x$host_summary_by_statement_type               OK
sys.x$innodb_buffer_stats_by_schema                OK
sys.x$innodb_buffer_stats_by_table                 OK
sys.x$innodb_lock_waits                            OK
sys.x$io_by_thread_by_latency                      OK
sys.x$io_global_by_file_by_bytes                   OK
sys.x$io_global_by_file_by_latency                 OK
sys.x$io_global_by_wait_by_bytes                   OK
sys.x$io_global_by_wait_by_latency                 OK
sys.x$latest_file_io                               OK
sys.x$memory_by_host_by_current_bytes              OK
sys.x$memory_by_thread_by_current_bytes            OK
sys.x$memory_by_user_by_current_bytes              OK
sys.x$memory_global_by_current_bytes               OK
sys.x$memory_global_total                          OK
sys.x$processlist                                  OK
sys.x$ps_digest_95th_percentile_by_avg_us          OK
sys.x$ps_digest_avg_latency_distribution           OK
sys.x$ps_schema_table_statistics_io                OK
sys.x$schema_flattened_keys                        OK
sys.x$schema_index_statistics                      OK
sys.x$schema_table_lock_waits                      OK
sys.x$schema_table_statistics                      OK
sys.x$schema_table_statistics_with_buffer          OK
sys.x$schema_tables_with_full_table_scans          OK
sys.x$session                                      OK
sys.x$statement_analysis                           OK
sys.x$statements_with_errors_or_warnings           OK
sys.x$statements_with_full_table_scans             OK
sys.x$statements_with_runtimes_in_95th_percentile  OK
sys.x$statements_with_sorting                      OK
sys.x$statements_with_temp_tables                  OK
sys.x$user_summary                                 OK
sys.x$user_summary_by_file_io                      OK
sys.x$user_summary_by_file_io_type                 OK
sys.x$user_summary_by_stages                       OK
sys.x$user_summary_by_statement_latency            OK
sys.x$user_summary_by_statement_type               OK
sys.x$wait_classes_global_by_avg_latency           OK
sys.x$wait_classes_global_by_latency               OK
sys.x$waits_by_host_by_latency                     OK
sys.x$waits_by_user_by_latency                     OK
sys.x$waits_global_by_latency                      OK
Phase 4/7: Running 'mysql_fix_privilege_tables'
ERROR 130 (HY000) at line 5: Incorrect file format 'column_stats'
ERROR 130 (HY000) at line 31: Incorrect file format 'help_topic'
ERROR 130 (HY000) at line 33: Incorrect file format 'help_relation'
ERROR 130 (HY000) at line 34: Incorrect file format 'help_keyword'
ERROR 130 (HY000) at line 35: Incorrect file format 'table_stats'
ERROR 130 (HY000) at line 36: Incorrect file format 'index_stats'
ERROR 130 (HY000) at line 58: Incorrect file format 'column_stats'
ERROR 130 (HY000) at line 77: Incorrect file format 'column_stats'
ERROR 130 (HY000) at line 146: Incorrect file format 'column_stats'
ERROR 130 (HY000) at line 164: Incorrect file format 'column_stats'
ERROR 130 (HY000) at line 237: Incorrect file format 'column_stats'
ERROR 130 (HY000) at line 299: Incorrect file format 'column_stats'
ERROR 130 (HY000) at line 366: Incorrect file format 'index_stats'
ERROR 130 (HY000) at line 505: Incorrect file format 'help_topic'
FATAL ERROR: Upgrade failed
mariadb 11:43:39.72 INFO  ==> Stopping mariadb
2022-12-25 11:43:39 0 [Note] /opt/bitnami/mariadb/sbin/mysqld (initiated by: unknown): Normal shutdown
2022-12-25 11:43:39 0 [Note] InnoDB: FTS optimize thread exiting.
2022-12-25 11:43:39 0 [Note] InnoDB: Starting shutdown...
2022-12-25 11:43:39 0 [Note] InnoDB: Dumping buffer pool(s) to /bitnami/mariadb/data/ib_buffer_pool
2022-12-25 11:43:39 0 [Note] InnoDB: Buffer pool(s) dump completed at 221225 11:43:39
2022-12-25 11:43:39 0 [Note] InnoDB: Removed temporary tablespace data file: "./ibtmp1"
2022-12-25 11:43:39 0 [Note] InnoDB: Shutdown completed; log sequence number 47867740; transaction id 28277
2022-12-25 11:43:39 0 [Note] /opt/bitnami/mariadb/sbin/mysqld: Shutdown complete

mariadb 11:43:40.73 DEBUG ==> Waiting for mariadb to unlock db files

To fix, firstly run a container with the same env vars that I made, but don't start the mysql/mariadb yet, it will fail because of the entrypoint script. Instead, create a bash shell so that we can work with mysql/mariadb.

docker run --rm -it -v "$(pwd)/data/mariadb:/bitnami/mariadb" \
 -e MARIADB_USER=bn_wordpress \
 -e MARIADB_DATABASE=bitnami_wordpress \
 -e MARIADB_PASSWORD=xxx \
 -e MARIADB_ROOT_PASSWORD=xxx \
 bitnami/mariadb:10.9 bash

Then, start mysql with a custom config, this will put innodb into recovery mode. Create a config file or append the current config file with this:

[mysqld]
innodb_force_recovery = 1

Note: for bitnami ijmage, the location of the binaries will be different, see reference: https://docs.bitnami.com/ibm/apps/wordpress/administration/recover-database-mariadb/

  • Config: /opt/bitnami/mariadb/my.cnf
  • Start script: /opt/bitnami/scripts/mariadb/run.sh

To start mariadb in background: /opt/bitnami/scripts/mariadb/run.sh &

Now we can work with mariadb. Let's login: mysql -u root -p

After logging in successfully, repair the tables:

  MariaDB> use mysql;
  MariaDB> repair table user use_frm;
  MariaDB> check table user use_frm;;
  MariaDB> exit;

Everything should be good now.

Initially, I failed to repair the table because I missed use_frm. Refer this: http://wepeex.blogspot.com/2010/09/repair-mysql-tables-with-error.html (archive link)

alias to take arguments

Feeling lucky

alias gg='f() { open "https://google.com/search?q=$*&btnI&sourceid=navclient&gfns=1" };f'

Python debugging profile, taking input args

github.com/microsoft/vscode/issues/83678#issuecomment-1266375549

{
    // Use IntelliSense to learn about possible attributes.
    // Hover to view descriptions of existing attributes.
    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Python: Current file debugging",
            "type": "python",
            "request": "launch",
            "program": "${file}",
            "console": "integratedTerminal",
            "justMyCode": true,
            "args": "${input:args}",
        },

    ],
    "inputs": [
        {
            "id": "args",
            "type": "promptString",
            "description": "Args for the program",
        }
    ]
}

Fast PgAdmin via Docker

Docker Compose

Create a file docker-compose.yml

version: '3.8'
services:
  pgadmin:
    container_name: pgadmin4_container
    image: dpage/pgadmin4
    restart: always
    network_mode: bridge
    volumes:
      - ./pgadmin4data:/var/lib/pgadmin
    environment:
      PGADMIN_DEFAULT_EMAIL: [email protected]
      PGADMIN_DEFAULT_PASSWORD: admin
    ports:
      - "5050:80"

Run & detach to background: docker compose up -d

Normal Docker container

If running normal docker container:

docker run --rm -d\
	--name pgadmin4_container \
	-e [email protected] \
	-e PGADMIN_DEFAULT_PASSWORD=admin \
	-v ${HOME}/sw/pgadmin/pgadmindata:/var/lib/pgadmin \
	-p 5050:80 \
	dpage/pgadmin4

PgAdmin

Open PgAdmin: http://localhost:5050

To connect to local DB running via Docker, we cannot use localhost, use the IP address from this command:

docker inspect daccel-test-postgres -f "{{json .NetworkSettings.Networks }}"

Default IP: 172.17.0.2

https://stackoverflow.com/questions/53610385/docker-postgres-and-pgadmin-4-connection-refused

Orico 3588US3 disconnects frequently

Orico 3588US3 uses ASMedia Technology Inc. ASM1051E SATA 6Gb/s bridge, ASM1053E SATA 6Gb/s bridge, ASM1153 SATA 3Gb/s bridge (174c:55aa), which is not compatible with UAS (USB Attached SCSI). So after a while, it will disconnect and you will have to remount it.

To solve this: blacklist it from using UAS.

echo options usb-storage quirks=174c:55aa:u | sudo tee /etc/modprobe.d/blacklist_uas.conf
sudo update-initramfs -u
sudo reboot

Notes

  • Somehow, it still disconnects for me, but after I setup a random shared folder on samba, the issue is gone??? (refer "Not used" part)
  • If you have other devices, check for the id with lsusb
  • Seems like ASM family has the same issues: ASM1053, ASM1051, ...

References:


Not used

Re: USB HDD disconnects
Sun Jul 28, 2019 5:45 am

At least Orico 3588US3-V1 has one ugly bug in the firmware:
The auto-shutdown feature ignores running reads. To prevent auto-shutdown while reading, you have to regularly write.
I am using following work around:
Code: Select all

while touch /run/media/sbrabec/Orico/temp_stamp ; do rm /run/media/sbrabec/Orico/temp_stamp ; sleep 20 ; done

Maybe it is possible to write a kernel quirk for that (without actually writing any data to the disk), but I am satisfied with this solution. When this script is running, Orico 3588US3-V1 works 100% reliably.

Quickly create venv for python

Run to add alias to ~/.zshrc

  echo 'alias venv="if [ -e ./venv/bin/activate ]; then source ./venv/bin/activate; else python3 -m venv venv && source ./venv/bin/activate; fi"' >> ~/.zshrc

Flare Mangadex@Home client issue with Oracle Cloud

sudo vi /etc/iptables/rules.v4
sudo iptables-restore </etc/iptables/rules.v4

This is rejecting the connection, need further investigation: -A INPUT -j REJECT --reject-with icmp-host-prohibited

To fix, add this line -A INPUT -p tcp --dport 44300 -j ACCEPT

Install docker (and compose) on ubuntu

1 copy

sudo apt-get update
 sudo apt-get install -y \
    ca-certificates \
    curl \
    gnupg \
    lsb-release
 sudo mkdir -p /etc/apt/keyrings
 curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
 echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
  $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
 sudo apt-get update
 sudo apt-get install -y docker-ce docker-ce-cli containerd.io docker-compose-plugin
 sudo usermod -aG docker $USER

Ref:

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.