Archive

Posts Tagged ‘computer’

Back to the grind – SSH Multiplexing – say wha?????

February 17, 2015 Leave a comment

So, It’s been quite a while since I posted anything on this blog. Thought I would get back into the swing of things.  I have tons of updates but lately very little time.   I think I’ll try to dedicate at least an hour a week to get something up here if not more.

For now, thought I would add to the latest linux tips n tricks I use to post.

SSH Multiplexing

I knew about this for quite a while but never really used it,  lately for the past few months it’s been a life saver. On avg during the course of my workday, I’m accessing 5-10 systems. This results to an ugly infestation of terminal windows, which surprisingly I have mastered the art of organizing thanks to tmux.

Anyways, what this allows you to do is use 1 TCP connection for each server you connect into.  Any additional SSH requests created from your client will use existing connection already established.  Cutting your login time in half.

Modify your existing ~/.ssh/config file and add the following:

Host *
   ControlMaster auto
   ControlPath ~/.ssh/master-%r@%h:%p

ControlMaster auto tells ssh to try to start a master if none is running, or to use an existing master otherwise. ControlPath is the location of a socket for the ssh processes to communicate among themselves. The %r, %h and %p are replaced with your user name, the host to which you’re connecting and the port number—only ssh sessions from the same user to the same host on the same port can or should share a TCP connection, so each group of multiplexed ssh processes needs a separate socket.

That’s it!!!

To test this, initiate a single ssh connection to any server. In a new window create another ssh connection to the same server. You should immediately see a difference the 2nd time around.

I would like to thank the Linux Journal for this cool tip, I had a 5 year old bookmark that I stumbled over a few months back that got me hooked.   You can find it Here

For now, I’m off, check back later.

Generating a clean MD5 Sum check file in python3

October 18, 2012 Leave a comment

This week I was stuck on a small problem involving me to generate hash sums for validation. Since I’ve been working on a automation project at work focusing on creating 100% hands free testing tools. A part of my test called for some basic procedures but I wanted to verify that the integrity of the data was sane. The original script that I was updating was written in Bash. Pretty much straight forward but yet, still room for improvement!  I decided to revise the script and convert it over to python, doing so would also make a more powerful tool giving the versatility that python has over simple bash scripts. src and dst are 2 parameters that are called elsewhere in the script. Essentially they are strings pointing to a path on your hard disk, for example dst = “/tmp/blah/blah/”


def prepimage(src, dst):
    '''
    Obtain sample files
    '''
    filename = "%s/md5sum.txt" % (dst)
    md5path = "%s/" % (dst)
    print ("Copying data and Generating md5sums")
    if not os.path.exists(dst):
        shutil.copytree(src, dst)

    #Generate md5sums
        list = subprocess.check_output(["ls", dst],universal_newlines=True)
        plist = list.split('\n')[0:2]
        f = open(filename, "wt")
        for item in plist:
           out = subprocess.Popen(['md5sum', item], universal_newlines=True\
                 , stdout=subprocess.PIPE, cwd='%s' % dst).communicate()[0]
           f.write(out)
        f.close()
    return 0

Lines 7 – 9 are simply creating my directory if it doesn’t exist.

We start @ Line 12,

list = subprocess.check_output(["ls", dst],universal_newlines=True)
        plist = list.split('\n')[0:2]

Using subprocess.Popen we kick of an “ls” command. dst is an argument that we set earlier pointing to a random directory. This will now give us output of something like this:

‘How fast.ogg\nJosh Woodward – Swansong.ogg\nmd5sum.txt\n’

the 2nd line will split the string using the delimiter \n to give us:

[‘How fast.ogg’, ‘Josh Woodward – Swansong.ogg’]

Now using this list, we can create a new file as I do in 14 and kick off a for loop to run md5sum against each entry in the list and write the output to our new file. The final output will look just like this:

cat /tmp/optical-test/Ubuntu_Free_Culture_Showcase/md5sum.txt 6e34a2a0eaa61748ba3a33015a84e813 How fast.ogg c9459a907b9345b289ba6c9e6517d4c2 Josh Woodward – Swansong.ogg

On the flip side, you can automate the integrity check by creating a new function and adding:


#Verify md5checksum
checkoutput = subprocess.Popen(['md5sum', '-c', 'md5sum.txt']
, universal_newlines=True, stdout=subprocess.PIPE\
, cwd='/media/CDROM/').communicate()[0]

Which should output:

How fast.ogg: OK

Josh Woodward – Swansong.ogg: OK

 

—————————————————————————————–

Python 3 also makes use of hashlib to generate the hashsum.. If you don’t need a checksum file then heres some alternate code you can use!

import hashlib
...
...
filename = "/tmp/file1.txt"
...
file = open(filename, 'rb')
filedata = file.read()
file.close()
md5 = hashlib.md5()
md5.update(filedata)
md5sum = md5.hexdigest()
print (md5sum)

This would generate just the hashsums:

6e34a2a0eaa61748ba3a33015a84e813
c9459a907b9345b289ba6c9e6517d4c2

Connecting to an IRC server behind a firewall? I feel your pain!

September 29, 2012 Leave a comment

So I use Xchat daily and connect to a private IRC server to talk with my colleagues. I also have a BIP server in the office to record all of the IRC transcripts, this way I never miss any conversations regardless of the time of day. Because the BIP server is behind a firewall on the companies network I can’t access it from the outside.  For the past year I’ve been working around this by connecting to my companies firewall via ssh and creating a SOCKS tunnel then simply directing xchat to talk through my local SOCKS proxy.

To do this ,  open a terminal and issue:

ssh -CND <LOCAL_IP_ADDRESS>:<PORT> <USER>@<SSH HOST>

Ex: ssh -CND 192.168.1.44:9999 sfeole@companyfirewall.com

Starting ssh with -CND:

‘D’ Specifies a local “dynamic” application-level port forwarding. This works by allocating a socket to listen to port on the local side, optionally bound to the specified bind_address. It also adds compression to the datastream ‘C’ and the ‘N’ is a safeguard which protects the user from executing remote commands.

192.168.1.44 is my  IPv4 address

9999 is the local port i’m going to open and direct traffic through

After the SSH tunnel is open I now need to launch xchat, navigate to Settings -> Preferences -> Network Setup and configure xchat to use my local IP (192.168.1.44) and local port (9999) then press OK then Reconnect.

I should now be able to connect to the IRC server behind the firewall. Usually I run through this process a few times a day, so it becomes somewhat of a tedious annoyance after a while.

Recently I finished a cool python3 script that does all of this in quick command.

The following code will do the following:

1.) identify the ipv4 address of the interface device you specify

2.) configure xchat.conf to use the new ipv4 address and port specified by the user

3.) open the ssh tunnel using the SSH -CND command from above

4.) launch xchat and connect to your server (assuming you have it set to auto connect)

To use it simply run

$./xchat.py -i <interface> -p <port>

ex: $./xchat.py -i wlan0 -p 9999

the user can select wlan0 or eth0 and of course their desired port. When your done with the tunnel simply issue <Ctrl-C> to kill it and wala!

https://code.launchpad.net/~sfeole/+junk/xchat

#!/usr/bin/env python3
#Sean Feole 2012,
#
#xchat proxy wrapper, for those of you that are constantly on the go:
#   --------------  What does it do? ------------------
# Creates a SSH Tunnel to Proxy through and updates your xchat config
# so that the user does not need to muddle with program settings

import signal
import shutil
import sys
import subprocess
import argparse
import re
import time

proxyhost = "myhost.company.com"
proxyuser = "sfeole"
localusername = "sfeole"

def get_net_info(interface):
    """
    Obtains your IPv4 address
    """

    myaddress = subprocess.getoutput("/sbin/ifconfig %s" % interface)\
                .split("\n")[1].split()[1][5:]
    if myaddress == "CAST":
        print ("Please Confirm that your Network Device is Configured")
        sys.exit()
    else:
        return (myaddress)

def configure_xchat_config(Proxy_ipaddress, Proxy_port):
    """
    Reads your current xchat.conf and creates a new one in /tmp
    """

    in_file = open("/home/%s/.xchat2/xchat.conf" % localusername, "r")
    output_file = open("/tmp/xchat.conf", "w")
    for line in in_file.readlines():
        line = re.sub(r'net_proxy_host.+', 'net_proxy_host = %s'
                 % Proxy_ipaddress, line)
        line = re.sub(r'net_proxy_port.+', 'net_proxy_port = %s'
                 % Proxy_port, line)
        output_file.write(line)
    output_file.close()
    in_file.close()
    shutil.copy("/tmp/xchat.conf", "/home/%s/.xchat2/xchat.conf"
                 % localusername)

def ssh_proxy(ProxyAddress, ProxyPort, ProxyUser, ProxyHost):
    """
    Create SSH Tunnel and Launch Xchat
    """

    ssh_address = "%s:%i" % (ProxyAddress, ProxyPort)
    user_string = "%s@%s" % (ProxyUser, ProxyHost)
    ssh_open = subprocess.Popen(["/usr/bin/ssh", "-CND", ssh_address,
                 user_string], stdout=subprocess.PIPE, stdin=subprocess.PIPE)

    time.sleep(1)
    print ("")
    print ("Kill this tunnel with Ctrl-C")
    time.sleep(2)
    subprocess.call("xchat")
    stat = ssh_open.poll()
    while stat is None:
        stat = ssh_open.poll()

def main():
    """
    Core Code
    """

    parser = argparse.ArgumentParser()
    parser.add_argument('-i', '--interface',
                        help="Select the interface you wish to use",
                        choices=['eth0', 'wlan0'],
                        required=True)
    parser.add_argument('-p', '--port',
                        help="Select the internal port you wish to bind to",
                        required=True, type=int)
    args = parser.parse_args()

    proxyip = (get_net_info("%s" % args.interface))
    configure_xchat_config(proxyip, args.port)
    print (proxyip, args.port, proxyuser, proxyhost)

    ssh_proxy(proxyip, args.port, proxyuser, proxyhost)

if __name__ == "__main__":
    sys.exit(main())

Refer to the launchpad address above for more info.

Ubuntu 12.10 Amazon Search, HOW TO REMOVE

September 24, 2012 1 comment

So it’s been quite a long time since my last post. Frankly I’ve been busy with work/personal life, spending time with my little girl and catching up on random chores in and out of the house. Everyone can relate I’m sure.

Well , so new info!! Some of you know that Ubuntu 12.10 Quantal Quetzal release is right around the corner. I’ve been using it since Beta1 and I have to say it’s coming along nicely. A new feature that has some users in an uproar is the Amazon Search feature in Dash. Whenever searching for new items using the Search Lens the system will now poll results from Amazon.com.

Here’s an example:

I search for Music and note the results from Amazon which populate in as I wait…

To Remove this, Open a Terminal Window <Ctrl-Alt-T>

$sudo apt-get remove unity-lens-shopping

Logout and LogBack then search for MUSIC again

All Fixed.

Hope this helps!! and Enjoy the new 12.10 Features!<

How to backup all my Windows pics to the Linux box

March 25, 2012 Leave a comment

So if your like me, then you shoot RAW!  I use Adobe Photoshop CS5 and Adobe Lightroom 3.5 for most of my post imaging which means my photography is mostly saved on a windows box. So there’s really no “play nice” backup solution. Plus I have about 20 some odd GB of photos which makes FTP out of the question.  What would be ideal is if they made rsync for windows….  Wait, they do!  Check it out here using Cygwin.

So assuming your using a file server like Ubuntu I’ll jot down some instructions so you can do this at home as well!

I’m already guessing that you have rsync installed on your linux box, if not that can be easily fixed by:

#sudo apt-get install rsync

  1. Create a file named rsyncd.conf in /etc
    1. #sudo vi /etc/rsyncd.conf
    2. Add the following to rsyncd.conf, replacing all instances of usernamewith your Ubuntu username:
      [usernamebackup]
      
          path = /home/username/backup
          comment = Backup
          uid = username
          gid = username
          read only = false
          auth users = username
          secrets file = /etc/rsyncd.secrets
    3. #sudo chmod 644 /etc/rsyncd.conf
  2. Create a file named rsyncd.secrets in /etc
    1. #sudo vi /etc/rsyncd.secrets
    2. Add the following to rsyncd.secrets, replacing username with your username and passwordwith a password of your choosing:
      username:password
    3. #sudo chmod 600 /etc/rsyncd.secrets
  3. Open rsync port by editing /etc/default/rsync and setting
    RSYNC_ENABLE=true
  4. Restart rsync
    #sudo /etc/init.d/rsync restart

Set up rsync client on Windows

  1. Install Cygwin, making sure  Net > rsync (3.0.8) and Net > openssh are selected
  2. Add C:\cygwin\bin;to the Windows PATH statement
    1. Right-click on My Computer and select Properties
    2. Switch to the Advanced tab and click the Environment Variables button at the bottom
    3. Find the “Path” or “PATH” variable in the System variables list at the bottom and click Edit
    4. Add C:\cygwin\bin; to the beginning of the list
  3. Create secret file to store password in Cygwin
      1. Start Cygwin Bash Shell
      2. Create secret file in the filesystem root and enter only the password in rsyncd.secrets above, with no spaces or line breaks
    #vi /secret
    #chmod 600 /secret
  4. Create bat file to run rsync
    1. Open Notepad and enter the following command, replacing User Name with your Windows User Name directory, username with your Ubuntu username, and ipaddress with the IP address of your Ubuntu server (e.g. 192.168.1.35):
      C:\cygwin\bin\rsync.exe --chmod u+rwx -qrtzv --password-file=/secret --delete "/cygdrive/d/photos" username@ipaddress::usernamebackup

      This will copy "/cygdrive/d/photos"  to the path specified in the rsyncd.conf. .

    2. Save the file as C:\rsync.bat

Now to simply kick off a backup just copy the batch file to your desktop and kick that off every time you finish your edits. You could also create a schedule task to execute the batch script daily @ midnight, I prefer to just run it manually as I don’t keep my linux box up or windows system 24/7

How to retrieve data after experiencing read errors on a USB stick

March 7, 2012 Leave a comment

So I figured I’d post this handy piece of info. Ever have to copy data from a USB stick and run into read errors.  I just did 😦

So aside from the fact I have some shitty USB keys, I really needed to save the data on this 4GB stick. I was able to retrieve the whole partition using gddrescue.  You can grab it from Ubuntu ppa..

sfeole@sfeolework:~$ apt-cache policy gddrescue
gddrescue:
Installed: 1.14-1
Candidate: 1.14-1
Version table:
*** 1.14-1 0
500 http://us.archive.ubuntu.com/ubuntu/ oneiric/universe amd64 Packages
100 /var/lib/dpkg/status

Read more about it here:

http://www.gnu.org/software/ddrescue/ddrescue.html

So by carrying out the following I was able to pull the partition off my USB stick, this copies the data block for block.

sfeole@sfeolework:~$ time sudo ddrescue -r3 -f /dev/sdb my4gusb.img
Press Ctrl-C to interrupt
rescued: 4012 MB, errsize: 0 B, current rate: 18481 kB/s
ipos: 4012 MB, errors: 0, average rate: 17958 kB/s
opos: 4012 MB, time from last successful read: 0 s
Finished

real 3m43.060s
user 0m0.308s
sys 0m13.773s

From here I was able to use a fresh new USB stick and DD my image back.

sfeole@sfeolework:~$dd if=my4gusb.img of=/dev/sdb

Where is Windows 8 going?

March 5, 2012 3 comments

So while going through my list of websites that I usually catch up on I was sent this link here regarding windows 8. I love the picture above btw, might to have to add it to my collection!

http://www.extremetech.com/computing/121015-windows-8-may-drive-me-to-linux

Now just by reading the title you would think that this is sort of a bashing the Window 8 OS itself, well quite frankly it is. It’s interesting the direction that microsoft is going. I’m not sure if everyone over there has their screws snugged tight, if you know what i mean.

May favorite pieces of this article:

Based on its current form, Windows 8 represents an unconscionable, and barely comprehensible, rejection of the values Microsoft has spent the last 26 years perfecting in its visual operating system.

And, of course, there are plenty of serious users who don’t want the PC on which they spend huge chunks of their waking life to look like it was designed by Fisher-Price.

I would have to agree with the Matthew Murray on this one, I’m really content with windows 7. In the event I want to play a game or two, i usually fire up my windows 7 box and game away. The world is not all tablets… yet???  hehe…  doubtful..  but again I would like to see how this pans out for microsoft, changing their operating environment so the user experience is the same across all devices desktops / tables & phones, I think they are a little late in the game for this. By the way, who in their right mind buys a Windows phone? 🙂

Anyways doesn’t all of this sound familiar now?  Ubuntu & Unity, the same desktop experience across desktops & phones & tablets, I don’t think that was announced as the original intent for switchig to Unity, but doesn’t it make sense now with the announcement of Ubuntu TV and Ubuntu on the Phone.   If I remember correctly, Ubuntu took their “fair share of flack” from the community for dropping gnome but that’s a story for another post. <Let the Linux Mint Trolls come out of their caves>

Ubuntu on the Atrix2