On Kubuntu Linux, The Google Chrome browser adds a checksum to the file, preventing simply editing the file by hand. So I'm writing a script to add the checksum.

$ cat .config/google-chrome/Default/Custom\ Dictionary.txt
AATEST
dotancohen
checksum_v1 = 2b7288da7c9556608de620e65308efa4$ 

No problem, I'll copy the entire file sans last line and check of its MD5 hash matches that checksum.

$ head -n -1 .config/google-chrome/Default/Custom\ Dictionary.txt > ~/chrome-dict
$ cat ~/chrome-dict
AATEST
dotancohen
$ md5sum ~/chrome-dict
2b7288da7c9556608de620e65308efa4  /home/dotancohen/chrome-dict

We got 2b7288da7c9556608de620e65308efa4, as expected. It matches! So let's add that to the end of the file.

$ { printf "checksum_v1 = " ; printf $(md5sum -z ~/chrome-dict | awk '{print $1}') ; } >> ~/chrome-dict
$ cat ~/chrome-dict
AATEST
dotancohen
checksum_v1 = 08f7dd79a17e12b178a1010057ef5e34$ 

No, wrong checksum! Lets try cat to ensure that nothing is written to the file between the two printf statements.

$ head -n -1 .config/google-chrome/Default/Custom\ Dictionary.txt > ~/chrome-dict
$ cat ~/chrome-dict
AATEST
dotancohen
$ { printf "checksum_v1 = " ; printf $(md5sum -z ~/chrome-dict | awk '{print $1}') ; } | cat >> ~/chrome-dict
$ cat ~/chrome-dict
AATEST
dotancohen
checksum_v1 = 08f7dd79a17e12b178a1010057ef5e34$ 

Still wrong checksum! Let's try a tmp file.

$ head -n -1 .config/google-chrome/Default/Custom\ Dictionary.txt > ~/chrome-dict
$ cat ~/chrome-dict
AATEST
dotancohen
$ { printf "checksum_v1 = " ; printf $(md5sum -z ~/chrome-dict | awk '{print $1}') ; } >> ~/chrome-dict-tmp
$ cat ~/chrome-dict-tmp >> ~/chrome-dict && rm ~/chrome-dict-tmp
$ cat ~/chrome-dict 
AATEST
dotancohen
checksum_v1 = 2b7288da7c9556608de620e65308efa4$ 

That worked! Why didn't the one-liners that redirect output to the end of the ~/chrome-dict file return the correct MD5 hash?

I have been running into problems with running out of disk space on an Ubuntu machine, so (after backing everything up!) I have been trying to find out what stuff that comes pre-installed with the Ubuntu 24 distribution can be safely removed. For example, I don't need it to print, and sudo apt remove --purge cups* and sudo apt remove --purge ghostscript go off without a hitch. Meanwhile, uninstalling Python causes an immediate crash, which I can only recover from by restoring the disk image from backup. And of course there are lots of installed packages whose purpose I am completely in the dark on.

So, what all can I safely remove? Or, how can I find out what I can safely remove, to minimize how much space is taken up just supporting the OS and Gnome environment, and maximize the space available for user data and programs?

I am working on creating a way to right click on an .eml file, right click, and have python pull two pieces of data from it and paste them into a new xlsx file and auto open the xlsx file.

I would try this code on a Linux but I don't know whether it would throw an error I can't fix.

I did the following on a mac and it failed to execute correctly.

A method is to have a shellscript in Automator call a .py file which calls a .ipynb file.

The action “Run Shell Script” encountered an error: When I run it on a .eml file I see, “/Applications/Xcode.app/Contents/Developer/usr/bin/python3: can't open file '/users///Documents/run_notebook1.py': [Errno 1] Operation not permitted”

When I use automator to test the script with "get specified finder items"

The action “Run Shell Script” encountered an error: “Python path configuration:
PYTHONHOME = (not set)
PYTHONPATH = (not set)
program name = '/Users//opt/anaconda3/bin/python'
isolated = 0
environment = 1
user site = 1
import site = 1
sys._base_executable = '/Users//opt/anaconda3/bin/python'
sys.base_prefix = '/Users//opt/anaconda3'
sys.base_exec_prefix = '/Users//opt/anaconda3'
sys.platlibdir = 'lib'
sys.executable = '/Users//opt/anaconda3/bin/python'
sys.prefix = '/Users//opt/anaconda3'
sys.exec_prefix = '/Users//opt/anaconda3'
sys.path = [
'/Users//opt/anaconda3/lib/python39.zip',
'/Users//opt/anaconda3/lib/python3.9',
'/Users//opt/anaconda3/lib/python3.9/lib-dynload',
]
Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
Python runtime state: core initialized
ModuleNotFoundError: No module named 'encodings'

>Current thread 0x00000001ef41cc00 (most recent call first):
<no Python frame>
Traceback (most recent call last):
File "/users//Documents/run_notebook1.py", line 27, in <module>
run_notebook(eml_file)
File "/users//Documents/run_notebook1.py", line 16, in run_notebook
subprocess.run(command, shell=True,check=True)
File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/subprocess.py", line 528, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '/Users//opt/anaconda3/bin/jupyter nbconvert --to notebook --execute --inplace --ExecutePreprocessor.timeout=-1 /Users//Documents/process_email1.ipynb' returned non-zero exit status 1.

Is the answer where is one is supposed to find python39.zip?

sys.path = [
'/Users//opt/anaconda3/lib/python39.zip',
'/Users//opt/anaconda3/lib/python3.9',
'/Users//opt/anaconda3/lib/python3.9/lib-dynload'

I ran the .py file in terminal using python3 file.py without problems.

Shellscript in Automator (xcode is installed so I don't need to specify python directory):

python3 /users//Documents/run_notebook1.py "$1"

The .py file:

import sys
import subprocess
import os

def run_notebook(eml_file):
    # Path to the notebook
    notebook_path = '/Users//Documents/process_email1.ipynb'

    # Set environment variable
    os.environ['EML_FILE_PATH'] = eml_file

    # Command to run the notebook with nbconvert
    command = f'/Users//opt/anaconda3/bin/jupyter nbconvert --to notebook --execute --inplace --ExecutePreprocessor.timeout=-1 {notebook_path}'

    # Run the command
    subprocess.run(command, shell=True,check=True)

# Ensure the file path is passed as argument
if len(sys.argv) != 2:
    print("Usage: python run_notebook.py <path_to_eml_file>")
    sys.exit(1)

# Get the EML file path
eml_file = sys.argv[1]

# Run the notebook with the EML file path
run_notebook(eml_file)

The .ipynb file

import email
import pandas as pd
import os
from email import policy
from email.parser import BytesParser
from datetime import datetime
import subprocess

# Retrieve the EML file path from the environment variable
eml_file = os.getenv('EML_FILE_PATH')

# Check if the environment variable is set
if not eml_file:
    raise ValueError("EML_FILE_PATH environment variable not set. Please pass the file path.")

# Function to extract email file information
def extract_eml_info(eml_file):
    # Parse the .eml file
    with open(eml_file, 'rb') as f:
        msg = BytesParser(policy=policy.default).parse(f)

    # Extract the file name (without extension)
    file_name = os.path.splitext(os.path.basename(eml_file))[0]

    # Extract the most recent date of the email conversation
    date_str = msg['date']
    if date_str:
        try:
            email_date = email.utils.parsedate_to_datetime(date_str)
        except Exception as e:
            email_date = None
    else:
        email_date = None

    return file_name, email_date

# Function to remove timezone information from DataFrame
def remove_timezone_from_df(df):
    # Iterate over all columns
    for col in df.columns:
        if pd.api.types.is_datetime64_any_dtype(df[col]):
            # Remove timezone info if it's a datetime column
            df[col] = df[col].dt.tz_localize(None)
    return df

# Function to process EML files
def process_eml_files(eml_files):
    # List to store extracted data
    data = []

    for eml_file in eml_files:
        file_name, email_date = extract_eml_info(eml_file)
        data.append([file_name, email_date])

    # Create DataFrame
    df = pd.DataFrame(data, columns=['File Name', 'Most Recent Date'])
    df = remove_timezone_from_df(df)

    # Write to Excel
    output_file = 'output_eml_data.xlsx'
    df.to_excel(output_file, index=False, engine='openpyxl')
    print(f"Data written to {output_file}")

    # Open the newly created Excel file
    if os.name == 'nt':  # For Windows
        os.startfile(output_file)
    elif os.name == 'posix':  # For macOS/Linux
        subprocess.call(['open', output_file])

    return output_file

# Process the provided EML file
process_eml_files([eml_file])

I just stumbled upon a weird behavior on android:

echo test>/data/local/tmp/test.txt
su root -c "cat /data/local/tmp/test.txt && \
    tar -cvzf /data/local/tmp/test.tar.gz /data/local/tmp/test.txt && \
    echo $?"
ls -alh /data/local/tmp/test.txt

This will produce an empty test.tar.gz:

test
removing leading '/' from member names
data/local/tmp/test.txt
0
-rw-r--r-- 1 root root 0 2025-01-09 17:43 /data/local/tmp/test.tar.gz

Running without su root -c I will get a 120 byte tar.gz:

-rw-r--r-- 1 root root 120 2025-01-09 17:47 /data/local/tmp/test.tar.gz

Adding a sleep like this:

su root -c "tar -cvzf /data/local/tmp/test.tar.gz /data/local/tmp/test.txt && sleep 1"

Also produces a 120 byte tar.gz.

Update:
Doing this with a 50mb file, the tar subcommand blocks until it's finished and produces an invalid tar.gz which is short of 22031 bytes to a valid tar.gz produced without su -c

$ ls -alh /system/apex/com.android.art.release.apex
-rw-r--r-- 1 root root 54M 2009-01-01 00:00 /system/apex/com.android.art.release.apex

$ tar -cvzf /data/local/tmp/big.tar.gz /system/apex/com.android.art.release.apex
$ su root -c "tar -cvzf /data/local/tmp/bigsu.tar.gz /system/apex/com.android.art.release.apex"
removing leading '/' from member names
system/apex/com.android.art.release.apex

$ ls -al /data/local/tmp/b*
-rw-r--r-- 1 root  root 22541839 2025-01-09 21:43 /data/local/tmp/big.tar.gz
-rw-r--r-- 1 root  root 22519808 2025-01-09 21:33 /data/local/tmp/bigsu.tar.gz

$ tar -xvzf /data/local/tmp/bigsu.tar.gz /data/local/tmp/big.apex ; echo $?
zcat: gzclose: Inappropriate ioctl for device
tar: EOF: Illegal seek
1

$ tar -xvzf /data/local/tmp/big.tar.gz /data/local/tmp/big.apex ; echo $?
0

Same behavior when using another user besides root.
This problem occurs on an android 11 emulator. I wasn't able to reproduce this on my linux servers.

The only way I can explain it is that the write buffer from tar is not flushed to the emulated disk.

A stat to the file inside the subcommand makes it flush the data correctly:

$ su root -c "tar -cvzf /data/local/tmp/test.tar.gz /data/local/tmp/test.txt && stat /data/local/tmp/test.tar.gz"
removing leading '/' from member names
data/local/tmp/test.txt
  File: /data/local/tmp/test.tar.gz
  Size: 120      Blocks: 16      IO Blocks: 512  regular file
Device: fd05h/64773d     Inode: 65546    Links: 1        Device 
type: 0,0
    Access: (0644/-rw-r--r--)       Uid: (    0/    root)   Gid: (    0/    root)
    Access: 2025-01-09 17:43:44.964000000 +0000
    Modify: 2025-01-09 22:00:28.496000000 +0000
    Change: 2025-01-09 22:00:28.496000000 +0000

$ ls -alh /data/local/tmp/test.tar.gz
-rw-r--r-- 1 root root 120 2025-01-09 22:00 /data/local/tmp/test.tar.gz

But why on earth would ending the subshell prevent flushing of the output file? When tar exits with a success exit 0, that means the written data should already be handled and flushed by the kernel.

Simple copying does not have the flush problem. So it might be a tar bug.

su root -c "cp /system/apex/com.android.art.release.apex>/data/local/tmp/release.apex"
su root -c "cat /system/apex/com.android.art.release.apex>/data/local/tmp/releasecat.apex"
ls -al /data/local/tmp/release*
-rw-r--r-- 1 root root 56510850 2025-01-09 22:31 /data/local/tmp/release.apex
-rw-r--r-- 1 root root 56510850 2025-01-09 22:33 /data/local/tmp/releasecat.apex

The tar version of busybox does not have this problem!

$ su root -c "/data/adb/magisk/busybox tar --version"
tar (busybox) 1.36.1-Magisk
$ su root -c "/data/adb/magisk/busybox tar -cvzf /data/local/tmp/test.tar.gz /data/local/tmp/test.txt"
$ ls -alh /data/local/tmp/test.tar.gz
-rw-r--r-- 1 root root 120 2025-01-09 22:42 /data/local/tmp/test.tar.gz

Notes:

• tar version toybox 0.8.3-android
• same when using /sdcard/Download as a directory.

My screen goes blank every 10 minutes (after idle time, not touching keyboard and touchpad).

I'd like to increase this time (40 minutes).

I tried to add this line to my .profile

xset s 2400

The second thing I tried is this command

gsettings set org.gnome.desktop.session idle-delay 2400

but unsuccessful

I'm on Ubuntu Ubuntu 22.04.4 LTS with i3

Here is the output of xset q

Keyboard Control:
  auto repeat:  on    key click percent:  0    LED mask:  00000000
  XKB indicators:
    00: Caps Lock:   off    01: Num Lock:    off    02: Scroll Lock: off
    03: Compose:     off    04: Kana:        off    05: Sleep:       off
    06: Suspend:     off    07: Mute:        off    08: Misc:        off
    09: Mail:        off    10: Charging:    off    11: Shift Lock:  off
    12: Group 2:     off    13: Mouse Keys:  off
  auto repeat delay:  660    repeat rate:  25
  auto repeating keys:  00fdffffdf7ffbbf
                        fedfffafffedffff
                        9fffffffffffffff
                        fff7ffffffffffff
  bell percent:  50    bell pitch:  400    bell duration:  100
Pointer Control:
  acceleration:  2/1    threshold:  4
Screen Saver:
  prefer blanking:  yes    allow exposures:  yes
  timeout:  2400    cycle:  600
Colors:
  default colormap:  0x20    BlackPixel:  0x0    WhitePixel:  0xffffff
Font Path:
  /usr/share/fonts/X11/misc,/usr/share/fonts/X11/Type1,built-ins
DPMS (Energy Star):
  Standby: 600    Suspend: 600    Off: 600
  DPMS is Enabled
  Monitor is On

I tried a few different terminal emulators on my Ubuntu 24.10. Specifically, Kitty, Ghostty and zellij. All three suddenly presented strange strange behavior: while I type, the cursor jumps a few spaces ahead (just as though I had pressed the TAB key), or it replaces what was typed with the newly typed character but without moving the cursor.

For example, when trying to type cd Media, I get something like this instead:

$ cd Med a 

Does anyone know how I can fix this?

I tried to do a fresh install of a Debian Bookworm on an Asus TUF Gaming A15 RTX 4070 (NVIDIA Optimus).
The installation of Debian went without issue. I use Xfce as the DE.
SecureBoot and Fastboot are disabled.

A first issue with the nouveau driver: black screen after the GRUB menu.
However using nomodeset allows me to start the computer and log in.

Afterwards I follow the instructions in the Debian wiki, That is:

• First install linux-headers-amd64
• Then install nvidia-driver-full and firmware-misc-nonfree
  • Actually it says nvidia-driver but I need the additional packages from nvidia-driver-full

And from there, nothing goes right.

• If I use nomodeset, LightDM fails to start
• If I use acpi=off, LightDM starts but I have no keyboard nor trackpad
• If I use nothing, I have a black screen after the GRUB menu, the same as with the nouveau driver

What am I missing?
I tried solutions here and there like using the kernel and NVIDIA driver from backports, other GRUB options, in vain.
I don't know where to start to do a correct debug as well.

I already managed to make it work on the same computer with a Debian Sid install, alas I fail to remember how!

VPS: running Ubuntu linux, accessed remotely via ssh.

I have a single ssl certificate, generated using Let's Encrypt. All is working fine.

This was before I added another domain, which is now also pointing, via DNS/nameservice, to this same VPS server's ip address.

Do I need to generate a separate ssl certificate for each domain-name?

I plan to have 2 separate websites, 1 for each domain name. And also a mail server that services both/each domain name's email requirements. Presumably I don't need separate certificates for mail servers and web servers, just the separate domains, if that.

The interface I am using is called usb0, and I would like to bring it up only on a certain interface.

I tried this:

ExecStartPre=/usr/lib/systemd/systemd-networkd-wait-online --interface=usb0

But it failed, and would not allow a connection. I then tried this, and got it to work reliably:

ExecStartPre=bash -c \
    'while ! (/usr/bin/netcat -v  -w 1 -t -l -p 23 192.168.7.2 2>&1 | grep Connection\ timed\ out) ; do \
         date ; \
         ((i++)); \
         ((i>60)) && echo "telnet.socket wait: Timer expire" && exit 1 ; \
     done;exit 0'

This is what I have that worked (seems like there should be a better way):

[Unit]
Description=Telnet Server Activation Socket
After=systemd-user-sessions.service

[Socket]
ListenStream=192.168.7.2:23
Accept=true
ExecStartPre=bash -c \
    'while ! (/usr/bin/netcat -v  -w 1 -t -l -p 23 192.168.7.2 2>&1 | grep Connection\ timed\ out) ; do \
         date ; \
         ((i++)); \
         ((i>60)) && echo "telnet.socket wait: Timer expire" && exit 1 ; \
     done;exit 0'

[Install]
WantedBy=multi-user.target

So here is another stab at this to eliminate socat. Since telnet.socket does not successfully start, it does not need to be enabled. Here is a simpler telnet.socket that also fails when enabled:

[Unit]
Description=Telnet Server Activation Socket

[Socket]
ListenStream=192.168.7.2:23
Accept=true

[Install]
WantedBy=sys-subsystem-net-devices-usb0.device

Now to fix this, I add and enable the following service:

[Unit]
Description=Get telnetd socket up

[Service]
ExecStart=/usr/libexec/telnetd-up/telnetd-up.sh
Type=oneshot
RemainAfterExit=yes
Nice=10

[Install]
WantedBy=sys-subsystem-net-devices-usb0.device

The telnetd-up.sh now merely needs to reliably start up the socket unit:

#!/bin/bash
while : ; do
    systemctl start telnet.socket
    if systemctl status telnet.socket 2>&1 >/dev/null  ; then
        exit 0
    fi
    systemctl stop telnet.socket
    sleep 1
done

NOTE: I realize telnet is not secure. This is an embedded device being booted from an image loaded serially through an external USB interface and this is not intended to be normal usage.

I've been using the iptables-save command recently (combined with iptables-restore) and noticed some inconsistent behaviour that I don't understand. I'm sure it's not a bug as it's far too popular a command for it to have gone unnoticed which means it's my own understanding that is flawed. I've looked over the man pages and various sources of documentation but have not been able to find anything that explains what I have observed.

The issue is that when I run the command iptables-save, I expect to see the filter table dumped to stdout (as per the documentation). What I in fact see, is absolutely nothing e.g:

[root@rhel9 ~]# iptables --list
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         
[root@rhel9 ~]# 
...
[root@rhel9 ~]# iptables-save
[root@rhel9 ~]# 

As you can see, there is nothing output to stdout from the command. Likewise, if I run the command and either try to redirect the output or pass the -f <output file> option, the output is still blank.

If I specify the table I wish to view, I do get output e.g:

[root@rhel9 ~]# iptables-save -t filter
# Generated by iptables-save v1.8.8 (nf_tables) on Wed Jan  8 16:43:11 2025
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
COMMIT
# Completed on Wed Jan  8 16:43:11 2025

If I save this output to a file, lets say ipv4backup and then call iptables-restore on that file, the behaviour of iptables-save is different:

[root@rhel9 ~]# iptables-restore ipv4backup
[root@rhel9 ~]# 
[root@rhel9 ~]# iptables --list
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         
[root@rhel9 ~]# 
[root@rhel9 ~]# iptables-save
# Generated by iptables-save v1.8.8 (nf_tables) on Wed Jan  8 16:44:03 2025
*filter
:INPUT ACCEPT [5:305]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [5:914]
COMMIT
# Completed on Wed Jan  8 16:44:03 2025

The tables contain the same content in the example above but there is different behaviour from iptables-save depending on whether iptables-restore was called beforehand.

I tested this on RHEL9 and Ubuntu 22.04 and both exhibit the same behaviour. On SuSE 15, the command ran and produced output of all tables regardless of whether iptables-restore had run previously. The equivalent ip6tables... commands exhibit the same behaviour.

So, the question is (based on the observed behaviour above), why does iptables-save not produce output when the tables are empty but if we restore empty tables and call the command again, we get the expected output (which is empty tables)?

On both pulseaudio and pipewire, there's a 1 second delay after audio files begin playing before they are audible. Afterwards, the audio has "woken up" and I can play audio without delay.

Audio devices: 00:1f.3 Audio device: Intel Corporation Raptor Lake High Definition Audio Controller (rev 11)

03:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Navi 31 HDMI/DP Audio

03:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Navi 32 [Radeon RX 7700 XT / 7800 XT] (rev ff)

Distro: Archcraft Linux

Monitor: ASUS VG279QM

I usually bind the § key (on my Swedish keyboard) to toggling my terminal window, on all my systems. I recently switched from KDE to XFCE on my Debian laptop and bound the § key to a bash script in the Keyboard part of the Settings Manager.

To my surprise this makes my W key stop responding. In the terminal pressing it makes the cursor flash briefly, in other programs nothing happens. Shift-W works and produces a W.

If I instead bind Shift-§ (it shows up as Shift-1/2 in the settings) I don't get this problem (not even with Shift-W).

Why does this happen, and more importantly, how can I get around it?

(I would prefer to not bind another key since this is the only key on my keyboard I don't use.)

Edit (in response to a question from @eyoung100):

The output from setxkbmap -query -v 10 is

Setting verbose level to 10
locale is C
Trying to load rules file ./rules/evdev...
Trying to load rules file /usr/share/X11/xkb/rules/evdev...
Success.
Applied rules from evdev:
rules:      evdev
model:      pc105
layout:     se
Trying to build keymap using the following components:
keycodes:   evdev+aliases(qwerty)
types:      complete
compat:     complete
symbols:    pc+se+inet(evdev)
geometry:   pc(pc105)
rules:      evdev
model:      pc105
layout:     se

Want to backup my datavg volume group to NAS drive using Redhat "Relax and Recover (ReaR)" tool. I need some help with the configuration file

#lsblk

sdb   8:16   0  100G  0 disk 
├─myvg-worktest2lg 253:5    0   10G  0 lvm  /root/work/test2 
└─myvg-test        253:6    0   10G  0 lvm  /work/test

When I explicitly mention the below line it backup all the files from “/root/work/test2” Not from “/work/test”. How can I backup all the files from myvg volume group at once.Is there any other parameter kindly suggest

BACKUP_PROG_INCLUDE=( '/root/work/test2' )

/etc/rear/local.conf

OUTPUT=ISO
OUTPUT_URL=nfs://mynasdrive/tmp/backup_dest
OUTPUT_PREFIX="$HOSTNAME.`date +%m%d%y`"
BACKUP=NETFS
BACKUP_URL=nfs://mynasdrive/tmp/backup_dest
NETFS_PREFIX="$HOSTNAME.`date +%m%d%y`"
ONLY_INCLUDE_VG=( "myvg" )
BACKUP_PROG_INCLUDE=( '/root/work/test2' )
BACKUP_OPTIONS="nfsvers=3,nolock"
OUTPUT_OPTIONS="nfsvers=3,nolock"
NETFS_KEEP_OLD_BACKUP_COPY=y

I've got a Western Digital Technologies, Inc. Elements 25A2. This is the way it introduces itself through the lsusb command.

Unfortunately, I cannot format it. Through Gnome Disk Utility the command refuses to run and returns: “Error wiping device: Failed to probe the device ‘/dev/sdd’ (udisks-error-quark, 0)”.

I tried various commands but they all failed:

• sudo fsck /dev/sdd
• sudo e2fsck -b 8193 /dev/sdd
• sudo e2fsck -b 32768 /dev/sdd

I want to recreate a file system because we can consider that the device is totally empty (and notice I haven't got any image copy).

(base) avy@machine:~$ sudo parted -l
Erreur: /dev/sdd : étiquette de disque inconnue #unknown disk's label
Modèle : WD Elements 25A2 (scsi)                                          
Disque /dev/sdd : 2000GB
Taille des secteurs (logiques/physiques) : 512B/512B #block's size (logical/physical)
Table de partitions : unknown #partition table
Drapeaux de disque : #disk's flags

As far as I know, solutions like DDRescue are based on existing image of a file system (e.g. sudo ddrescue <file_name_source> <file_name_target>.img <file_name_log>), so I can’t use them.

The hard drive itself seems "healthy" after SMART control:

• sudo smartctl --health /dev/sdd → SMART overall-health self-assessment test result: PASSED
• sudo smartctl --log=error /dev/sdd → SMART Error Log Version: 1 \n No Errors Logged

So, I have a little hope to avoid throwing it in the trash.

Here are the kernel messages relating to the drive:

sudo dmesg --follow

[  444.527131] usb 2-4: Product: Elements 25A2
[  444.527134] usb 2-4: Manufacturer: Western Digital
[  444.527138] usb 2-4: SerialNumber: 575855314533383859304150
[  444.528739] usb-storage 2-4:1.0: USB Mass Storage device detected
[  444.529073] scsi host6: usb-storage 2-4:1.0
[  445.546672] scsi 6:0:0:0: Direct-Access     WD       Elements 25A2    1021 PQ: 0 ANSI: 6
[  445.546937] sd 6:0:0:0: Attached scsi generic sg3 type 0
[  445.547812] sd 6:0:0:0: [sdd] Spinning up disk...
[  446.570416] ........ready
[  453.738963] sd 6:0:0:0: [sdd] 3906963456 512-byte logical blocks: (2.00 TB/1.82 TiB)
[  453.739246] sd 6:0:0:0: [sdd] Write Protect is off
[  453.739251] sd 6:0:0:0: [sdd] Mode Sense: 47 00 10 08
[  453.739479] sd 6:0:0:0: [sdd] No Caching mode page found
[  453.739487] sd 6:0:0:0: [sdd] Assuming drive cache: write through
[  456.492061] sd 6:0:0:0: [sdd] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[  456.492069] sd 6:0:0:0: [sdd] tag#0 Sense Key : Medium Error [current] 
[  456.492074] sd 6:0:0:0: [sdd] tag#0 Add. Sense: Unrecovered read error
[  456.492080] sd 6:0:0:0: [sdd] tag#0 CDB: Read(10) 28 00 00 00 00 00 00 00 08 00
[  456.492087] blk_update_request: critical medium error, dev sdd, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
[  456.492096] Buffer I/O error on dev sdd, logical block 0, async page read
[  459.561974] sd 6:0:0:0: [sdd] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[  459.561982] sd 6:0:0:0: [sdd] tag#0 Sense Key : Medium Error [current] 
[  459.561988] sd 6:0:0:0: [sdd] tag#0 Add. Sense: Unrecovered read error
[  459.561994] sd 6:0:0:0: [sdd] tag#0 CDB: Read(10) 28 00 00 00 00 00 00 00 08 00
[  459.562001] blk_update_request: critical medium error, dev sdd, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
[  459.562010] Buffer I/O error on dev sdd, logical block 0, async page read
[  462.747978] sd 6:0:0:0: [sdd] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[  462.747986] sd 6:0:0:0: [sdd] tag#0 Sense Key : Medium Error [current] 
[  462.747991] sd 6:0:0:0: [sdd] tag#0 Add. Sense: Unrecovered read error
[  462.747997] sd 6:0:0:0: [sdd] tag#0 CDB: Read(10) 28 00 00 00 00 00 00 00 08 00
[  462.748004] blk_update_request: critical medium error, dev sdd, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
[  462.748012] Buffer I/O error on dev sdd, logical block 0, async page read
[  462.748032] ldm_validate_partition_table(): Disk read failed.
[  465.948181] sd 6:0:0:0: [sdd] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[  465.948188] sd 6:0:0:0: [sdd] tag#0 Sense Key : Medium Error [current] 
[  465.948192] sd 6:0:0:0: [sdd] tag#0 Add. Sense: Unrecovered read error
[  465.948197] sd 6:0:0:0: [sdd] tag#0 CDB: Read(10) 28 00 00 00 00 00 00 00 08 00
[  465.948202] blk_update_request: critical medium error, dev sdd, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
[  465.948210] Buffer I/O error on dev sdd, logical block 0, async page read
[  469.216369] sd 6:0:0:0: [sdd] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[  469.216371] sd 6:0:0:0: [sdd] tag#0 Sense Key : Medium Error [current] 
[  469.216372] sd 6:0:0:0: [sdd] tag#0 Add. Sense: Unrecovered read error
[  469.216374] sd 6:0:0:0: [sdd] tag#0 CDB: Read(10) 28 00 00 00 00 00 00 00 08 00
[  469.216375] blk_update_request: critical medium error, dev sdd, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
[  469.216378] Buffer I/O error on dev sdd, logical block 0, async page read
[  472.382131] sd 6:0:0:0: [sdd] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[  472.382139] sd 6:0:0:0: [sdd] tag#0 Sense Key : Medium Error [current] 
[  472.382145] sd 6:0:0:0: [sdd] tag#0 Add. Sense: Unrecovered read error
[  472.382151] sd 6:0:0:0: [sdd] tag#0 CDB: Read(10) 28 00 00 00 00 00 00 00 08 00
[  472.382157] blk_update_request: critical medium error, dev sdd, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
[  472.382166] Buffer I/O error on dev sdd, logical block 0, async page read
[  475.548568] sd 6:0:0:0: [sdd] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[  475.548570] sd 6:0:0:0: [sdd] tag#0 Sense Key : Medium Error [current] 
[  475.548571] sd 6:0:0:0: [sdd] tag#0 Add. Sense: Unrecovered read error
[  475.548573] sd 6:0:0:0: [sdd] tag#0 CDB: Read(10) 28 00 00 00 00 00 00 00 08 00
[  475.548574] blk_update_request: critical medium error, dev sdd, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
[  475.548577] Buffer I/O error on dev sdd, logical block 0, async page read
[  475.548611] Dev sdd: unable to read RDB block 0
[  478.628200] sd 6:0:0:0: [sdd] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[  478.628208] sd 6:0:0:0: [sdd] tag#0 Sense Key : Medium Error [current] 
[  478.628213] sd 6:0:0:0: [sdd] tag#0 Add. Sense: Unrecovered read error
[  478.628220] sd 6:0:0:0: [sdd] tag#0 CDB: Read(10) 28 00 00 00 00 00 00 00 08 00
[  478.628227] blk_update_request: critical medium error, dev sdd, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
[  478.628236] Buffer I/O error on dev sdd, logical block 0, async page read
[  481.707768] sd 6:0:0:0: [sdd] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[  481.707776] sd 6:0:0:0: [sdd] tag#0 Sense Key : Medium Error [current] 
[  481.707782] sd 6:0:0:0: [sdd] tag#0 Add. Sense: Unrecovered read error
[  481.707788] sd 6:0:0:0: [sdd] tag#0 CDB: Read(10) 28 00 00 00 00 00 00 00 08 00
[  481.707795] blk_update_request: critical medium error, dev sdd, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
[  481.707804] Buffer I/O error on dev sdd, logical block 0, async page read
[  484.864341] sd 6:0:0:0: [sdd] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[  484.864349] sd 6:0:0:0: [sdd] tag#0 Sense Key : Medium Error [current] 
[  484.864355] sd 6:0:0:0: [sdd] tag#0 Add. Sense: Unrecovered read error
[  484.864361] sd 6:0:0:0: [sdd] tag#0 CDB: Read(10) 28 00 00 00 00 18 00 00 08 00
[  484.864368] blk_update_request: critical medium error, dev sdd, sector 24 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
[  484.864377] Buffer I/O error on dev sdd, logical block 3, async page read
[  487.982899] sd 6:0:0:0: [sdd] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[  487.982907] sd 6:0:0:0: [sdd] tag#0 Sense Key : Medium Error [current] 
[  487.982912] sd 6:0:0:0: [sdd] tag#0 Add. Sense: Unrecovered read error
[  487.982919] sd 6:0:0:0: [sdd] tag#0 CDB: Read(10) 28 00 00 00 00 00 00 00 08 00
[  487.982925] blk_update_request: critical medium error, dev sdd, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
[  487.982935] Buffer I/O error on dev sdd, logical block 0, async page read
[  491.116325] sd 6:0:0:0: [sdd] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[  491.116333] sd 6:0:0:0: [sdd] tag#0 Sense Key : Medium Error [current] 
[  491.116339] sd 6:0:0:0: [sdd] tag#0 Add. Sense: Unrecovered read error
[  491.116345] sd 6:0:0:0: [sdd] tag#0 CDB: Read(10) 28 00 00 00 00 00 00 00 08 00
[  491.116351] blk_update_request: critical medium error, dev sdd, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
[  491.116361] Buffer I/O error on dev sdd, logical block 0, async page read
[  491.116432]  sdd: unable to read partition table
[  491.504986] sd 6:0:0:0: [sdd] Attached SCSI disk
[  494.284826] sd 6:0:0:0: [sdd] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[  494.284834] sd 6:0:0:0: [sdd] tag#0 Sense Key : Medium Error [current] 
[  494.284840] sd 6:0:0:0: [sdd] tag#0 Add. Sense: Unrecovered read error
[  494.284846] sd 6:0:0:0: [sdd] tag#0 CDB: Read(10) 28 00 00 00 00 00 00 00 08 00
[  494.284853] blk_update_request: critical medium error, dev sdd, sector 0 op 0x0:(READ) flags 0x80700 phys_seg 1 prio class 0
[  497.472400] sd 6:0:0:0: [sdd] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[  497.472408] sd 6:0:0:0: [sdd] tag#0 Sense Key : Medium Error [current] 
[  497.472413] sd 6:0:0:0: [sdd] tag#0 Add. Sense: Unrecovered read error
[  497.472419] sd 6:0:0:0: [sdd] tag#0 CDB: Read(10) 28 00 00 00 00 00 00 00 08 00
[  497.472426] blk_update_request: critical medium error, dev sdd, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
[  497.472435] Buffer I/O error on dev sdd, logical block 0, async page read

Disclaimer: I won't accept answers such as "Buy a new hard-drive", "Use warranty" (all examples given are true stories). There's nothing of value in this hard-drive, I want to improve my IT skills and bring added value to Stack Exchange platforms.

We have requirement to normalize the data ... Item field is comma delimited and irregular and it may have any items from 0 to max (lets say 100)

Input:

key1|desc field|item1,item2,item3,item4|extra field
key2|desc field|item1,item2,item3|extra field

Output:

key1|desc field|item1|extra field
key1|desc field|item2|extra field
key1|desc field|item3|extra field
key1|desc field|item4|extra field
key2|desc field|item1|extra field
key2|desc field|item2|extra field
key2|desc field|item3|extra field

is there a way we can achive it through Unix command or shell script? Please advise.

Since Fedora 41, the following command does not work anymore:

$ sudo dnf history userinstalled
Unknown argument "userinstalled" for command "history".

The list of Changes between DNF and DNF5 does not mention this change.

Using dnf, how do you list the packages that have been explicitly installed by the user?

I am trying to keep running of a command after logout from ssh. I used the commands nohup source a.sh > a.txt and screen source a.sh > a.txt. But both gives

nohup: failed to run command 'source': No such file or directory```
error. In one question, it is said that use the absolute path of command but I couldn't find the exact location of source command. But I can use nohup directly with other commands such as cp. How can I solve this issue or are there better alternatives?

Scenario

Have an external device, encrypted with LUKS, automatically mount when inserted and not cause any issues if not present on boot.

Most questions on this topic refer to mounting on boot - this is NOT what I am trying to do

Additionally, the mountpoint should be protected from accidental writes when the usb device is not present and I can do this with chattr +i <mountpoint>

Problem

1. When the external device is NOT mounted, any attempt to access the mountpoint will hang until timeout - this can cause system instability.

2. When the device is inserted, the passphrase is prompted for and device unlocked, but not mounted. I then have to mount the device manually with mount /mnt/backup or mount /dev/mapper/fit which appears to correctly use the fstab settings.

Configuration

# /etc/crypttab
fit   UUID=xxxxxxxx-xxxxx-xxxxxxxx-xxxxxxxxxx none luks,noauto

# /etc/fstab
/dev/mapper/fit    /mnt/backup   ext4    noatime,user,noauto,x-systemd.automount,x-systemd.device-timeout=5ms,x-systemd.mount-timeout=100ms    0   0

I happen to have a Samsung Fit usb flash drive if anyone was wondering about the name

Accessing the mountpoint when the external device is NOT present

# Accessing the mountpoint when the drive is NOT plugged in
andy@pop-os:mnt$ ll
ls: cannot access 'backup': No such device
total 0
d????????? ? ? ? ?            ? backup/

andy@pop-os:mnt$ mountpoint /mnt/backup
/mnt/backup is a mountpoint

# journalctl
Jan 21 16:33:34 pop-os systemd[1]: mnt-backup.automount: Got automount request for /mnt/backup, triggered by 5192 (ls)
Jan 21 16:33:34 pop-os systemd[1]: dev-mapper-fit.device: Job dev-mapper-fit.device/start timed out.
Jan 21 16:33:34 pop-os systemd[1]: Timed out waiting for device /dev/mapper/fit.
Jan 21 16:33:34 pop-os systemd[1]: Dependency failed for /mnt/backup.
Jan 21 16:33:34 pop-os systemd[1]: mnt-backup.mount: Job mnt-backup.mount/start failed with result 'dependency'.
Jan 21 16:33:34 pop-os systemd[1]: dev-mapper-fit.device: Job dev-mapper-fit.device/start failed with result 'timeout'.

Normal Removal Processes

# Absolute path is required if not using sudo (`user` was set in fstab)
andy@pop-os:mnt$ umount /mnt/backup

andy@pop-os:mnt$ sudo cryptsetup close fit

andy@pop-os:mnt$ sudo eject /dev/sdx

Temporary Resolution

I can unmount the mountpoint (even though no device is mounted anyway) to temporarly fix this issue but it will come back after a system reboot or after mounting/ummounting the device again.

andy@pop-os:mnt$ sudo umount backup

andy@pop-os:mnt$ ll
total 4.0K
drwxr-xr-x 2 root root 4.0K Jan 19 10:16 backup/

andy@pop-os:mnt$ lsattr
----i---------e------- ./backup

andy@pop-os:~$ mountpoint /mnt/backup
/mnt/backup is not a mountpoint

If I use noauto and don't include x-systemd.automount then I avoid the problem of the directory being a mountpoint on boot when no device is present, but I also don't get automounting - although the device still unlocks.

Mounting the external device

Passphrase is prompted for and device unlocked

andy@pop-os:mnt$ lsblk -f
NAME            FSTYPE      FSVER    LABEL     UUID                                   FSAVAIL FSUSE% MOUNTPOINTS
sdb             crypto_LUKS 2                  xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx                  
└─fit           ext4        1.0                yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy                  

andy@pop-os:mnt$ ll backup/
ls: cannot access 'backup/': No such device

Note that MOUNTPOINTS is empty

# journalctl -f
Jan 21 17:17:26 pop-os kernel: usb 6-2: new SuperSpeed USB device number 2 using xhci_hcd
Jan 21 17:17:26 pop-os kernel: usb 6-2: New USB device found, idVendor=090c, idProduct=1000, bcdDevice=11.00
Jan 21 17:17:26 pop-os kernel: usb 6-2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
Jan 21 17:17:26 pop-os kernel: usb 6-2: Product: Flash Drive FIT
Jan 21 17:17:26 pop-os kernel: usb 6-2: Manufacturer: Samsung
Jan 21 17:17:26 pop-os kernel: usb 6-2: SerialNumber: 0123456789123
Jan 21 17:17:26 pop-os kernel: usb-storage 6-2:1.0: USB Mass Storage device detected
Jan 21 17:17:26 pop-os kernel: scsi host7: usb-storage 6-2:1.0
Jan 21 17:17:26 pop-os mtp-probe[10452]: checking bus 6, device 2: "/sys/devices/pci0000:00/0000:00:08.1/0000:0b:00.4/usb6/6-2"
Jan 21 17:17:26 pop-os mtp-probe[10452]: bus: 6, device: 2 was not an MTP device
Jan 21 17:17:26 pop-os mtp-probe[10467]: checking bus 6, device 2: "/sys/devices/pci0000:00/0000:00:08.1/0000:0b:00.4/usb6/6-2"
Jan 21 17:17:26 pop-os mtp-probe[10467]: bus: 6, device: 2 was not an MTP device
Jan 21 17:17:29 pop-os kernel: scsi 7:0:0:0: Direct-Access     Samsung  Flash Drive FIT  1100 PQ: 0 ANSI: 6
Jan 21 17:17:29 pop-os kernel: sd 7:0:0:0: Attached scsi generic sg1 type 0
Jan 21 17:17:29 pop-os kernel: sd 7:0:0:0: [sdb] 501253132 512-byte logical blocks: (257 GB/239 GiB)
Jan 21 17:17:29 pop-os kernel: sd 7:0:0:0: [sdb] Write Protect is off
Jan 21 17:17:29 pop-os kernel: sd 7:0:0:0: [sdb] Mode Sense: 43 00 00 00
Jan 21 17:17:29 pop-os kernel: sd 7:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesnt support DPO or FUA
Jan 21 17:17:29 pop-os kernel:  sdb: sdb1
Jan 21 17:17:29 pop-os kernel: sd 7:0:0:0: [sdb] Attached SCSI removable disk
Jan 21 17:17:41 pop-os systemd[1]: Starting Cryptography Setup for fit...
Jan 21 17:17:41 pop-os systemd-cryptsetup[10585]: Volume fit already active.
Jan 21 17:17:41 pop-os systemd[1]: Finished Cryptography Setup for fit.
Jan 21 17:17:41 pop-os systemd[1]: Reached target Block Device Preparation for /dev/mapper/fit.
Jan 21 17:17:41 pop-os gnome-shell[3744]: Unable to mount volume 257 GB Encrypted: Gio.IOErrorEnum: Operation was cancelled
Jan 21 17:17:41 pop-os udisksd[1382]: Unlocked device /dev/sdb1 as /dev/dm-4

Note that the second to last line "Operation was cancelled" is happening just before udisksd reports "Unlock device..."

I've tried this with two different devices, one with whole disk block encryption and the other with an encrypted partition, it makes no difference.

Sometimes moments later...

andy@pop-os:mnt$ mount /mnt/backup
mount: /mnt/backup: /dev/mapper/fit already mounted on /mnt/backup.

... but usually this just mounts the device as expected.

Other Issues

• If I do not set x-systemd.device-timeout to something much less than the default 90s then my system will slowly become unresponsive and then crash. I thought systemd might be timing out waiting for me to type in my passphrase but longer timeouts didn't seem to help
• I copied the configuration from an old Ubuntu 22.04 system to Pop!_OS 22.04, I would sometimes see the question marks while listing the directory in the unmounted state but the auto-mount worked perfectly. On the old system I didn't set the immutable attribute on the mountpoint.

Related

• mount-error-when-automounting-a-luks-encrypted-usb-flashdrive - different errors and method of mounting
• Automounting USB sticks on Debian - uses a slightly different method with nofail instead of noauto but unfortunately doesn't answer my question
• I can get more information with systemctl show mnt-backup.mount but this hasn't helped me
• The man page for fstab says what to do for external devices but these settings don't auto mount the device and do cause the boot process to hang

So, why does accessing the mountpoint try to access the device after it has been unmounted resulting in ?????????, and why does it not automount correctly when the device is inserted? Between the old fstab and new systemd I can't seem to find the magic variables.

Would also be useful to know why the system crashes if I don't set the device timeout?

Update August 2023

mount-vs-automount-systemd-units... explains the ????? as the .automount unit attempts to open the mountpoint on access.

I have still been unable to achieve the behaviour I want though, and wonder if it because without configuring anything udev is used and then the only problem is being able to customise the directory used as a mountpoint. As soon as an fstab entry is made, /run/systemd/generator systemd units are made and the behaviour changes.

There also seems to be a problem on my new system with plymouth as this spits out errors while systemd uses 100% CPU and hangs the system. This doesn't happen on my old laptop.

I'm using LXC containers, and resolving CONTAINERNAME.lxd to the IP of the specified container, using:

sudo resolvectl dns lxdbr0 $bridge_ip
sudo resolvectl domain lxdbr0 '~lxd'

This works great! But the changes don't persist over a host reboot - how can I make them do so?

I'm on Pop!_OS 22.04, which is based on Ubuntu 22.04.

(I've described 'things I've tried' as answers to this question, which have varying degrees of success.)

I've just launched "virt-manager" on my Ubuntu 21.04 and with my big surprise I've seen the error "kvm is not available. It could means that the kvm package is not installed and so on" ; this is not true. I've installed kvm and it is working great,as u can see the image below :

give a look at what the terminal is telling : "/dev/kvm exists ; kvm acceleration can be used". What Am I supposed to believe ? What's missing ?

# apt install qemu-kvm libvirt-clients libvirt-daemon-system

Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Note, selecting 'qemu-system-x86' instead of 'qemu-kvm'
qemu-system-x86 is already the newest version (1:5.2+dfsg-9ubuntu3.1).

libvirt-clients is already the newest version (7.0.0-2ubuntu2).
libvirt-daemon-system is already the newest version (7.0.0-2ubuntu2).
0 upgraded, 0 newly installed, 0 to remove and 11 not upgraded.

# virt-host-validate

  QEMU: Checking for hardware virtualization                                 : PASS
  QEMU: Checking if device /dev/kvm exists                                   : PASS
  QEMU: Checking if device /dev/kvm is accessible                            : PASS
  QEMU: Checking if device /dev/vhost-net exists                             : PASS
  QEMU: Checking if device /dev/net/tun exists                               : PASS
  QEMU: Checking for cgroup 'cpu' controller support                         : PASS
  QEMU: Checking for cgroup 'cpuacct' controller support                     : PASS
  QEMU: Checking for cgroup 'cpuset' controller support                      : PASS
  QEMU: Checking for cgroup 'memory' controller support                      : PASS
  QEMU: Checking for cgroup 'devices' controller support                     : PASS
  QEMU: Checking for cgroup 'blkio' controller support                       : PASS
  QEMU: Checking for device assignment IOMMU support                         : PASS
  QEMU: Checking if IOMMU is enabled by kernel                               : PASS
  QEMU: Checking for secure guest support                                    : WARN (Unknown if this platform has Secure Guest support)
   LXC: Checking for Linux >= 2.6.26                                         : PASS
   LXC: Checking for namespace ipc                                           : PASS
   LXC: Checking for namespace mnt                                           : PASS
   LXC: Checking for namespace pid                                           : PASS
   LXC: Checking for namespace uts                                           : PASS
   LXC: Checking for namespace net                                           : PASS
   LXC: Checking for namespace user                                          : PASS
   LXC: Checking for cgroup 'cpu' controller support                         : PASS
   LXC: Checking for cgroup 'cpuacct' controller support                     : PASS
   LXC: Checking for cgroup 'cpuset' controller support                      : PASS
   LXC: Checking for cgroup 'memory' controller support                      : PASS
   LXC: Checking for cgroup 'devices' controller support                     : PASS
   LXC: Checking for cgroup 'freezer' controller support                     : PASS
   LXC: Checking for cgroup 'blkio' controller support                       : PASS
   LXC: Checking if device /sys/fs/fuse/connections exists                   : PASS

Again :

● libvirtd.service - Virtualization daemon
     Loaded: loaded (/lib/systemd/system/libvirtd.service; enabled; vendor preset: enabled)
     Active: active (running) since Wed 2021-07-28 13:18:31 CEST; 41min ago
TriggeredBy: ● libvirtd-admin.socket
             ● libvirtd-ro.socket
             ● libvirtd.socket
       Docs: man:libvirtd(8)
             https://libvirt.org
   Main PID: 2091 (libvirtd)
      Tasks: 21 (limit: 32768)
     Memory: 49.3M
     CGroup: /system.slice/libvirtd.service
             ├─2091 /usr/sbin/libvirtd
             ├─2405 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/lib/libvirt/libvirt_leaseshelper
             └─2406 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/lib/libvirt/libvirt_leaseshelper

Jul 28 12:45:30 Z390-AORUS-PRO libvirtd[2091]: internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-x86_64 for probing: libvirt:  error : cannot execute binary /usr/local/bin/qemu-system-x86_64: Permission denied
Jul 28 12:45:30 Z390-AORUS-PRO libvirtd[2091]: Failed to probe capabilities for /usr/local/bin/qemu-system-x86_64: internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-x86_64 for probing: libvirt:  error : cannot execute binary /usr/local/bin/qemu-system-x86_64: Permission denied
Jul 28 13:58:54 Z390-AORUS-PRO libvirtd[2091]: internal error: libxenlight state driver is not active
Jul 28 13:58:54 Z390-AORUS-PRO libvirtd[2091]: End of file while reading data: Input/output error
Jul 28 13:58:54 Z390-AORUS-PRO libvirtd[2091]: invalid argument: could not find capabilities for arch=x86_64 domaintype=qemu 
Jul 28 13:58:54 Z390-AORUS-PRO libvirtd[2091]: internal error: Cannot find suitable emulator for x86_64
Jul 28 13:58:54 Z390-AORUS-PRO libvirtd[2091]: internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-x86_64 for probing: libvirt:  error : cannot execute binary /usr/local/bin/qemu-system-x86_64: Permission denied
Jul 28 13:58:54 Z390-AORUS-PRO libvirtd[2091]: Failed to probe capabilities for /usr/local/bin/qemu-system-x86_64: internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-x86_64 for probing: libvirt:  error : cannot execute binary /usr/local/bin/qemu-system-x86_64: Permission denied
Jul 28 13:59:06 Z390-AORUS-PRO libvirtd[2091]: internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-x86_64 for probing: libvirt:  error : cannot execute binary /usr/local/bin/qemu-system-x86_64: Permission denied
Jul 28 13:59:06 Z390-AORUS-PRO libvirtd[2091]: Failed to probe capabilities for /usr/local/bin/qemu-system-x86_64: internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-x86_64 for probing: libvirt:  error : cannot execute binary /usr/local/bin/qemu-system-x86_64: Permission denied

My terminal emulators are lagging occasionally. It's not heavy a lag. May be once in 20-30 mins( I think). The terminal becomes unresponsive and all the characters I typed during lag appears suddenly. It's not happening in any other app (even in SMPLAYER during 1080p video playback). The lag occurs even during low memory usage(nearly 1 GB free). I thought there might be a problem in xterm and switched to xfce4-terminal emulator. problem still persisted. Then thought there might be a bug in bash and switched to zsh . still no luck. What's happening ? How can I narrow the down problem?
system info : Arch + i3 + compton
Update1: I thought my history size(1000) might be causing the problem and changed it to 100. And this doesn't seem to work either.
Update2: My ~/.bashrc

#
# ~/.bashrc
#

# If not running interactively, don't do anything
[[ $- != *i* ]] && return

alias ls='ls --color=auto'
alias grep='grep --color=auto'

PS1='\[\033[32m\]\u\[\033[33m\]@\[\033[36m\]\h \[\033[31m\]\W\[\033[33m\]\$\[\033[00m\]'


## my settings
alias vi='vim'
alias vi_i3='vim ~/.config/i3/config'
alias pacs='sudo pacman -S'
alias pacss='pacman -Ss'
alias pacsyu='sudo pacman -Syu'
export TERMINAL='xfce4-terminal'
HISTSIZE=100

Update3: But in zsh I used very simple Prompt string without any color. Still it was lagging.

I'm running ISC DHCPd on Raspbian. I want to set up my server so that all addresses in 192.168.0.0/24 are static leases, and all in 192.168.1.0/24 are dynamic leases. I'd like the two to share the 192.168.0.0/23 subnet.

Clarification: Basically, I want a single DHCP server to handle DHCP for the entire 192.168.0.0/23 subnet. If a device that requests an IP has a host entry defined in dhcpd.conf, it will assign it the IP address that is specified. If it does not, it will assign an address in the range 192.168.1.2 - 192.168.1.254. For simplicity, I have limited all of my static leases to reside in the range 192.168.0.2 - 192.168.0.254.

I have my server configured as so:

option domain-name "home";
option domain-name-servers 192.168.0.3;
default-lease-time 600;
max-lease-time 7200;
ddns-update-style none;
authoritative;
host mightyturing{ hardware ethernet 10:bf:48:87:74:a2; fixed-address 192.168.0.64; }
host beastnugget{ hardware ethernet 34:e6:d7:52:1e:e4; fixed-address 192.168.0.65; }
host ... (about 50 more static leases)
subnet 192.168.1.0 netmask 255.255.255.0 {
        option routers 192.168.0.1;
        option broadcast-address 192.168.1.255;
        option domain-name-servers 192.168.0.3, 1.1.1.1;
        option domain-name "home";
        option subnet-mask 255.255.254.0;
        ddns-updates off;
}

When I try to start the service it whines:

Nov 17 17:45:26 raspberrypi dhcpd[681]: DHCPREQUEST for 192.168.0.64 from 10:bf:48:87:74:a2 via eth0
Nov 17 17:45:26 raspberrypi dhcpd[681]: DHCPACK on 192.168.0.64 to 10:bf:48:87:74:a2 via eth0
Nov 17 17:45:59 raspberrypi dhcpd[681]: Dynamic and static leases present for 192.168.0.64.
Nov 17 17:45:59 raspberrypi dhcpd[681]: Remove host declaration mightyturing or remove 192.168.0.64
Nov 17 17:45:59 raspberrypi dhcpd[681]: from the dynamic address pool for 192.168.0.0/23

... and since my lease pool is 192.168.1.0/24... I'm pretty sure that means that 192.168.0.64 is not in the pool. So why is ISC not recognizing this? Why is it complaining about a range that I have not instructed it to issue dynamic IPs for?

Also:

When I change the range to issue dynamic IPs on the 10.0.0.0/24 range (to test ISC's sanity), but keep the static leases in 192.168.0.0/24, it still complains about 192.168.0.0/23 so what is going on? Where, precisely, did I ask it to hand out 192.168.0.0/23 addresses?

option domain-name "home";
option domain-name-servers 192.168.0.3;
default-lease-time 600;
max-lease-time 7200;
ddns-update-style none;
authoritative;
host mightyturing{ hardware ethernet 10:bf:48:87:74:a2; fixed-address 192.168.0.64; }
host beastnugget{ hardware ethernet 34:e6:d7:52:1e:e4; fixed-address 192.168.0.65; }
subnet 10.0.0.0 netmask 255.255.255.254 {
        option routers 10.0.0.1;
        option broadcast-address 10.0.0.255;
        option domain-name-servers 1.1.1.1;
        option domain-name "home";
        option subnet-mask 255.255.255.0;
        ddns-updates off;
}

I install Debian a lot. To do this I have a fully-automated preseed.cfg; at the end of the preseed, it downloads and runs a postinstall.sh script from my TFTP server, which does some additional customization.

I'm in the process of switching from GNOME to LXQTE, and using SDDM instead of GDM. However, SDDM tries to start too quickly for my hardware. To get around this, I've been using systemctl edit sddm to add the following:

[Service]
ExecStartPre=/bin/sleep 5

This works great, and I'd like to automate this process by adding it to the postinstall.sh script. However, I can't figure out how to pass the file contents to systemctl edit via a bash script. How can I do this?

i want to test several linux VPS using bench mark tools as i reead there are 2 indestry standart tools called unixBench and SysBench I compiled them and executed them on the VPS And i have results : SysBench:( 4 CPU)

./sysbench --test=cpu --cpu-max-prime=20000 --num-threads=4 run

The result :

General statistics:
    total time:                          3.222s
    total number of events:              10000

Latency (ms):
         min:                                  1.64
         avg:                                  5.76
         max:                                  6.19
         95th percentile:                      3.00
         sum:                              60000.86

Threads fairness:
    events (avg/stddev):           30000.0000/2.00
    execution time (avg/stddev):   8.0002/0.00

from reading i know that the important info is in : total time: 3.222s ok .. but compared to what ? how can i know that this is good result ? also what about the other parameters ? like 95th percentile what does it means ?

Now running UnixBench ( 4 CPU )

./Run -c 4

The result :

BYTE UNIX Benchmarks (Version 5.1.3)

   System: ip-10-0-1-48: GNU/Linux
   OS: GNU/Linux -- 3.14.48-33.39.amzn1.x86_64 -- #1 SMP Tue Jul 14 23:43:07 UTC 2015
   Machine: x86_64 (x86_64)
   Language: en_US.UTF-8 (charmap="UTF-8", collate="UTF-8")
   CPU 0:  info .. 
   CPU 1:  info .. 
   CPU 2:  info .. 
   CPU 3:  info .. 


------------------------------------------------------------------------
Benchmark Run: Wed Apr 12 2017 
4 CPUs in system; running 4 parallel copies of tests

Dhrystone 2 using register variables       74325935.8 lps   (10.0 s, 7 samples)
Double-Precision Whetstone                    13710.8 MWIPS (9.9 s, 7 samples)
Execl Throughput                               3528.0 lps   (30.0 s, 2 samples)
File Copy 1024 bufsize 2000 maxblocks        422092.9 KBps  (30.0 s, 2 samples)
File Copy 256 bufsize 500 maxblocks          107334.5 KBps  (30.0 s, 2 samples)
File Copy 4096 bufsize 8000 maxblocks       1485937.1 KBps  (30.0 s, 2 samples)
Pipe Throughput                              998109.2 lps   (10.0 s, 7 samples)
Pipe-based Context Switching                 162959.5 lps   (10.0 s, 7 samples)
Process Creation                               7151.7 lps   (30.0 s, 2 samples)
Shell Scripts (1 concurrent)                   6494.3 lpm   (60.0 s, 2 samples)
Shell Scripts (8 concurrent)                    880.4 lpm   (60.1 s, 2 samples)
System Call Overhead                         900145.3 lps   (10.0 s, 7 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   74325935.8   6369.0
Double-Precision Whetstone                       55.0      13710.8   2492.9
Execl Throughput                                 43.0       3528.0    820.5
File Copy 1024 bufsize 2000 maxblocks          3960.0     422092.9   1065.9
File Copy 256 bufsize 500 maxblocks            1655.0     107334.5    648.5
File Copy 4096 bufsize 8000 maxblocks          5800.0    1485937.1   2562.0
Pipe Throughput                               12440.0     998109.2    802.3
Pipe-based Context Switching                   4000.0     162959.5    407.4
Process Creation                                126.0       7151.7    567.6
Shell Scripts (1 concurrent)                     42.4       6494.3   1531.7
Shell Scripts (8 concurrent)                      6.0        880.4   1467.3
System Call Overhead                          15000.0     900145.3    600.1
                                                                   ========
System Benchmarks Index Score                                        1157.3 

Here again i know i should look at the : System Benchmarks Index Score 1157.3

but again the question raised this result is compared to what ? how should i know if this total result is good ? bad ? average ? Thanks

I the old days, resolv.conf was static and you edited it yourself.

Later on, the DHCP client would rewrite it, using some static entries and what it got from the DHCP lease.

These days, some distributions - like mine (Linux Mint 18.1) have a /etc/resolv.conf.d mechanism with several subdirectories of scripts, and, well, I can't make heads and tails of it.

What's the right way, then, to...

• indicate that I want to use or not-use the DNS server(s) obtained in the DHCP lease?
• add fixed entries to resolv.conf, always or as a fallback when there's no DHCP-lease-obtained nameserver?
• make decisions about relative order in the file?

I don't mind writing some scripts of my own if I have to, I just don't want to "work against" the existing mechanism or do duplicate work.

I need to rewrite a domain name in a query parameter in order to augment a proxy. The proxy basically is just a domain rewrite, but some of the responses from the upstream server include a query parameter that is eventually used as a redirect (in the "Location" header). I need to be able to rewrite this domain name so that it points to the proxy instead of the upstream server. My current config looks like

<VirtualHost *:443>

SSLEngine On
SSLProxyEngine On
# Set the path to SSL certificate
# Usage: SSLCertificateFile /path/to/cert.pem
SSLCertificateFile /etc/apache2/ssl/apache.crt
SSLCertificateKeyFile /etc/apache2/ssl/apache.key

# Servers to proxy the connection, or;
# List of application servers:
# Usage:
# ProxyPass / http://[IP Addr.]:[port]/
# ProxyPassReverse / http://[IP Addr.]:[port]/
# Example:

RewriteEngine On
LogLevel alert rewrite:trace6
RewriteCond %{QUERY_STRING} (.*)upstream\.com(.*)
RewriteRule (.*)upstream\.com(.*)  %1proxy.com%2 [N]

ProxyPass / https://upstream.com/
ProxyPassReverse / https://upstream.com/

</VirtualHost>

The proxy is working fine, but one of the first things the upstream server does is send a redirect with a query parameter that is used for a second redirect, which causes the proxy to lose control of the traffic stream. I need this query string rewritten so the upstream servers response will cause a redirect back to the proxy instead of the upstream server.

mod_rewrite is enabled and 'apachectl -t' reports that the config is OK.

As part of a research I am doing I am trying to install TOR package on a Amazon ec2 server.

I have added the repo name to my repos configuration (following this tutorial: https://www.torproject.org/docs/rpms.html.en)

When attempting to install the tor package using sudo yum install tor I get the following error:

> Loaded plugins: priorities, update-motd, upgrade-helper
amzn-main/latest                                         | 2.1 kB     00:00
amzn-updates/latest                                      | 2.3 kB     00:00
Resolving Dependencies
--> Running transaction check
---> Package tor.x86_64 0:0.2.7.6-tor.1.rh7_1_1503 will be installed
--> Processing Dependency: openssl-libs >= 1.0.1 for package: tor-0.2.7.6-tor.1.rh7_1_1503.x86_64
--> Finished Dependency Resolution
**Error: Package: tor-0.2.7.6-tor.1.rh7_1_1503.x86_64 (tor)
           Requires: openssl-libs >= 1.0.1**
 You could try using --skip-broken to work around the problem
 You could try running: rpm -Va --nofiles --nodigest

And so I tried to install openssl and I found out that it is already installed.

Has anyone encountered this problem before? Any solutions would be a great help! Thank you in advance!

I just wrote a bash script and always getting this EOF-Error.

So here is my script (only works on OS X):

#!/bin/bash

#DEFINITIONS BEGIN
en_sq() {
    echo -e "Enabling smart quotes..."
    defaults write NSGlobalDomain NSAutomaticQuoteSubstitutionEnabled -bool true
    status=$(defaults read NSGlobalDomain NSAutomaticQuoteSubstitutionEnabled -bool)
            if [ "$status" = "1" ]
                then
                    echo -e "Success! Smart quotes are now enabled."
                    SUCCESS="TRUE"
            else
                echo -e "Sorry, an error occured. Try again."
            fi
}
di_sq() {
    echo -e "Disabling smart quotes..."
    defaults write NSGlobalDomain NSAutomaticQuoteSubstitutionEnabled -bool false
    status=$(defaults read NSGlobalDomain NSAutomaticQuoteSubstitutionEnabled -bool)
            if [ "$status" = "0" ]
                then
                    echo -e "Success! Smart quotes are now disabled."
                    SUCCESS="TRUE"
            else
                echo -e "Sorry, an error occured. Try again."
            fi
}
en_sd() {
    echo -e "Enabling smart dashes..."
    defaults write NSGlobalDomain NSAutomaticDashSubstitutionEnabled -bool true
    status=$(defaults read NSGlobalDomain NSAutomaticDashSubstitutionEnabled -bool)
            if [ "$status" = "1" ]
                then
                    echo -e "Success! Smart dashes are now enabled."
                    SUCCESS="TRUE"
            else
                echo -e "Sorry, an error occured. Try again."
            fi
}
di_sd() {
    echo -e "Enabling smart dashes..."
    defaults write NSGlobalDomain NSAutomaticDashSubstitutionEnabled -bool false
    status=$(defaults read NSGlobalDomain NSAutomaticDashSubstitutionEnabled -bool)
            if [ "$status" = "0" ]
                then
                    echo -e "Success! Smart dashes are now disabled."
                    SUCCESS="TRUE"
            else
                echo -e "Sorry, an error occured. Try again."
            fi
}
#DEFINITIONS END
#---------------

#BEGIN OF CODE with properties
#This is only terminated if the user entered properties (eg ./sqd.sh 1 1)
if [ "$1" = "1" ]
    then
        en_sq
    elif [ "$1" = "0" ]
        then
            di_sq
fi

if [ "$2" = "1" ]
    then
        en_sd
        #exit 0 if both, $1 and $2 are correct entered and processed.
        exit 0
    elif [ "$1" = "0" ]
        then
            di_sd
            #exit 0 if both, $1 and $2 are correct entered and processed.
            exit 0
fi
#END OF CODE with properties
#---------------------------


#BEGIN OF CODE without properties
#This is terminated if the user didn't enter two properties
echo -e "\n\n\n\n\nINFO: You can use this command as following: $0 x y, while x and y can be either 0 for false or 1 for true."
echo -e "x is for the smart quotes, y for the smart dashes."
sleep 1
echo -e " \n Reading preferences...\n"
status=$(defaults read NSGlobalDomain NSAutomaticQuoteSubstitutionEnabled -bool)
if [ "$status" = "1" ]
    then
        echo -e "Smart quotes are enabled."
    elif [ "$status" = "0" ]
    then
        echo -e "Smart quotes are disabled."

    else
        echo -e "Sorry, an error occured. You have to run this on OS X""
fi

status=$(defaults read NSGlobalDomain NSAutomaticQuoteSubstitutionEnabled -bool)
if [ "$status" = "1" ]
    then
        echo -e "Smart dashes are enabled."
    elif [ "$status" = "0" ]
    then
        echo -e "Smart dashes are disabled."

    else
        echo -e "Sorry, an error occured. You have to run this on OS X!"
fi

sleep 3
echo -e "\n\n You can now enable or disable smart quotes."

until [ "$SUCCESS" = "TRUE" ]
do
echo -e "Enter e for enable or d for disable:"
read sq

if [ "$sq" = "e" ]
    then
        en_sq
    elif [ "$sq" = "d" ]
        then
            di_sq
    else
        echo -e "\n\n ERROR! Please enter e for enable or d for disable!"
fi
done
SUCCESS="FALSE"

echo -e "\n\n You can now enable or disable smart dashes."

until [ "$SUCCESS" = "TRUE" ]
do
echo -e "Enter e for enable or d for disable:"
read sq

if [ "$sd" = "e" ]
    then
        en_sd
    elif [ "$sd" = "d" ]
        then
            di_sd
    else
        echo -e "\n\n ERROR! Please enter e for enable or d for disable!"
fi
done

And here is my error:

./coding.sh: line 144: unexpected EOF while looking for matching `"'
./coding.sh: line 147: syntax error: unexpected end of file

I am using debian 7.4 cd. When I set a partition as '/' during manual partition. It is fine. However when I then try to use it as 'physical volume for encryption'...everything works fine it erases and writes random data BUT THEN mount point '/' disappears! . And when I try to proceed with installation,it says "NO root file system is defined.Please correct this from partitioning menu"

And when I try to make '/', I cant modify the partition to be used as '/' because it says-"No modification can be made.In use as physical volume for encrypted volume"

I am following these instructions:http://www.debianuserforums.org/viewtopic.php?f=9&t=460

Could I get ZFS to work properly in Linux?

Are there any caveats / limitations?