Paramiko/Open SSH get env variable paths

See also questions close to this topic

• SSH access to github repo on codeship

  I am attempting to push to github from a container on Codeship. After getting a Permission denied (publickey) error, I followed the suggestion here:

  https://documentation.codeship.com/pro/builds-and-configuration/setting-ssh-private-key/

  I created a service called publish to and some steps to try to recreate the article's suggestion.

  My codeship_services.yml file:

  # codeship_services.yml
  publish:
    build: 
      image: codeship/setting-ssh-key-test
      dockerfile: Dockerfile.publish
    encrypted_env_file: codeship.env.encrypted
    volumes:
      - ./.ssh:/root/.ssh
  

  My codeship_steps.yml file:

  - name: temp publish service
    service: publish
    command: /bin/bash -c "echo -e $PRIVATE_SSH_KEY >> /root/.ssh/id_rsa"
  
  - name: chmod id_rsa
    service: publish
    command: chmod 600 /root/.ssh/id_rsa
  
  - name: add server to list of known hosts
    service: publish
    command: /bin/bash -c "ssh-keyscan -H github.com >> /root/.ssh/known_hosts"
  
  - name: confirm ssh connection to server, authenticating with generated public ssh key
    service: publish
    command: /bin/bash -c "ssh -T git@github.com"
  
  

  When running jet steps, however, I still get the Permission denied (publickey) error:

  (step: temp_publish_service) success ✔
  (step: chmod_id_rsa) 
  (step: chmod_id_rsa) success ✔
  (step: add_server_to_list_of_known_hosts) 
  (service: publish) (step: add_server_to_list_of_known_hosts) # github.com:22 SSH-2.0-babeld-80573d3e
  (service: publish) (step: add_server_to_list_of_known_hosts) # github.com:22 SSH-2.0-babeld-80573d3e
  (service: publish) (step: add_server_to_list_of_known_hosts) # github.com:22 SSH-2.0-babeld-80573d3e
  (step: add_server_to_list_of_known_hosts) success ✔
  (step: confirm_ssh_connection_to_server,_authenticating_with_generated_public_ssh_key) 
  (service: publish) (step: confirm_ssh_connection_to_server,_authenticating_with_generated_public_ssh_key) Permission denied (publickey).
  (step: confirm_ssh_connection_to_server,_authenticating_with_generated_public_ssh_key) error ✗
  (step: confirm_ssh_connection_to_server,_authenticating_with_generated_public_ssh_key) container exited with a 255 code
  

  I have generated the keys as instructed in the article and added the encrypted private key to codeship.env.encryped.

  Is there something I am missing?
• sshpass from iplist and run tcptraceroute

  Number of servers in a list = 40

  I want to run tcptraceroute on each server for all 40 ips in the list. for this i need to loop sshpass and run tcptracetraceroute.

  When i run the below code, it just runs for first ip in list and use same ip to run tcptraceroute and exits.

  IFS=$IFS,
    USER='*********'
    PASSWORD='********'
    PORT='22'
      while read ip; do
          sshpass -p $PASSWORD ssh -i turbot -t -o StrictHostKeyChecking=no 
          $USER@$ip "sudo -s /usr/bin/tcptraceroute "$ip" $PORT" >>Res.txt
  
      done < .PrivateIP-List.txt
  

  I'm trying to output the traceroute results to Res.txt file
• How to specify remote shell in fabric 2.4

  I need to connect via ssh to a remote host from python. I've chosen fabric 2.4 because it can run multiple commands in the single ssh-session. But I need to use a remote shell different from sh/bash/etc, my shell is powered by clixon.

  All examples I've found described changing shell in fabric 1.X.

  How can I configure it in fabric 2.4?

  Or maybe you can advice another ssh library for python that can run multiple commands in single ssh-session?

  P.S. I can't change the default shell for user in /etc/passwd.
• Error in cmd: "'javac' is not recognized as an internal or external command, operable program or batch file."

  I'm trying to execute java programs. How to overcome the below error in cmd? 'javac' is not recognized as an internal or external command, operable program or batch file.

  Followed following steps:
  1. Install jdk 12.0.1
  2. Under Advanced system settings, environment variables, system variable,
  (i)add

  Variable name: JAVA_HOME Variable value: C:\Program Files\Java\jdk-12.0.1

  (ii)Edit path variable: C:\Program Files\Java\jdk-12.0.1\bin

  C:\Users\HP>javac -version 'javac' is not recognized as an internal or external command, operable program or batch file.
• How does my production Heroku app know what secret key to use if I am not supposed to commit .env?

  In my local file structure, I created a file called .env to hold my secret key for JWT authentication. When I deploy my app to Heroku using git and then try to send the API requests using Postman, I get this message:

  Error: secretOrPrivateKey must have a value

  From other searches it appears this is because I didn't deploy my .env file where that info is located. However, the web also makes it clear you are not supposed to deploy that file.

  Which is why I ask, how on earth is my app supposed do encode the JWT without having the secret key in that .env available?
• Gitlab Docker build: calling shell command in .gitlab-ci.yml

  I'm trying to call shell command in .gitlab-ci.yml, whose relevant parts are:

  image: docker:latest
  services:
    - docker:dind
  
  stages:
    - build
    - deploy
  
  ...
  
  build:
    stage: build
    script:
      - apt-get update -y
      - GIT_TAG=$(git tag | tail -1)
      - GIT_TAG=$(/usr/bin/git tag | tail -1)
      - docker ...
  

  However all top three shell command callings have failed, all with "command not found" error. The git command being failing is really odd, because it has to get the git repo first before start the script section. I.e., I can see that git is working, but I just can't use it myself.

  Is there any way to make it working?
• unable to download files from sftp server using python

  i am writing a python script that helps me download files from sftp server to my local folder. when i run the script it just downloads the blank document i keep on trying but i am failing

  i made a sftp server to test the code and the path specified in remote path is server root directory

  i giving code below

  import pysftp
  
  myHostname = "192.168.56.1"
  myUsername = "new45"
  myPassword = "146515"
  
  cnopts = pysftp.CnOpts()
  cnopts.hostkeys = None
  
  with pysftp.Connection(host=myHostname, username=myUsername, password=myPassword,cnopts=cnopts) as sftp:
      # Define the file that you want to download from the remote directory
      remoteFilePath = 'C:\\Users\\Simon\\Desktop\\ftp\\testfile.txt'
  
      # Define the local path where the file will be saved
      # or absolute "C:\Users\sdkca\Desktop\TUTORIAL.txt"
      localFilePath = 'C:\\Users\\Simon\\Desktop\\ftp2\\textfile.txt'
  
      sftp.get(remoteFilePath, localFilePath)
  
  # connection closed automatically at the end of the with-block'''
  

  please tell me whats the error why is blank file is being downloaded
• The tar.gz file transfer via SCP client cause corruption of file
  1. Compress several directories from Linux server via tar.gz
  2. Download compressed tar.gz file from server to Windows computer.
  3. Try to untar file via pythons's tarfile module
  4. Process pops one of Empty file (Fail to untar)

  I need to make tar file in sever cause I need to transfer lots of small files(most of them are less than a kilobyte). So I try to 1) Compress files to tar.gz file from server 2) transfer via SCP1 client 3) remove tar file from server(if needed) 4) Extract downloaded file inside the my python program. 5) Create excel statistics.

  I checked tar.gz file from serverside and I'am sure that file is not corrupted(I mean it is compressed well). It is extracted well without any error if I extract them inside the server ssh. But it pops an error above when I transfer tar.gz file from server via scp client inside the my program. And when I transfer file manually using FileZilla and extract using gitbash, it is not corrupted.

  I checked many threads on the Internet, they usually say that it is scp binary mode problem. But I'm not sure what should I do to solve this problem.

  I use scp and paramiko for the libaray. And this transfering phase is responsible for scp module. (I heared that it is re-created scp client module stems from paramiko

  import paramiko
  from scp import SCPClient
  ... # (Other class functions)
      def downloadCompressedFile(self, remote_paths, save_path):
          # binarial only
          # remote_paths :: Files to be tared
          # save_path :: Local path to be downloaded
          try:
              print('Compression Targets -->\n{}'.format(', '.join(remote_paths)))
              conn = self.getSSHConnection()
              tar_save_path = '{}/{}.tar.gz'.format(ROOT_TAR_PATH, datetime.now().strftime('%Y%m%d_%H%M%S'))
              obj = [ '-C {} ..'.format(p) for p in remote_paths]
              command = 'tar cvzf {} {}'.format(tar_save_path, ' '.join(obj))
              print('Remote Command -- tar -cvzf {} {}'.format(tar_save_path, ' '.join(obj)))
              conn.exec_command(command=command)
              print('Compressions are done. Downloading files from {} to {}'.format(tar_save_path, save_path))
              with SCPClient(conn.get_transport()) as scp:
                  scp.get(remote_path=tar_save_path, local_path=save_path)
  
          except Exception as e:
              raise Exception(e)
  ...
  

  It should transfer uncorrupted file.
• Not able to do nested ssh using ssh_key in paramiko

  I am trying to nested SSH using paramiko where I will connect to system A and from there I will connect to system B. Here to connect to System A I am using username, password authentication and to connect to system B using username, ssh_key. The connection to System A is good but while trying to communicate to SystemB I am getting the error

  client = paramiko.SSHClient()
          client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
          client.connect(sys_A, 22, user, password)
          print "Logged successfully to System A"
          #
  
          sshtransport = client.get_transport()
          dest_addr = (sys_B, 22)  # edited#
          local_addr = (sys_A, 22)  # edited#
          sshchannel = sshtransport.open_channel("direct-tcpip", dest_addr, local_addr)
          #
          print "Logging to the node given"
          node_client = paramiko.SSHClient()
          node_client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
          k = paramiko.RSAKey.from_private_key_file('/home/test_user/test/ssh_key')
          node_client.connect(sys_B, username='root', pkey=k, sock=sshchannel)
  
          print "Login successful to the node"
          #
          stdin, stdout, stderr = node_client.exec_command(cmd)  # edited#
          stdoutstr = ''
          stderrstr = ''
          for line in stdout.readlines():
              stdoutstr = stdoutstr + line + '\n'
          for line in stderr.readlines():
              stderrstr = stderrstr + line + '\n'
          return (stdoutstr, stderrstr)
  

  And I am getting following error:

  ----------------------------------------------------------------------
  Traceback (most recent call last):     
    File "isolated_vm_test_base.py", line 173, in ssh_lpar_node_and_run_command
      k = paramiko.RSAKey.from_private_key_file('/home/test_user/test/id_rsa')
    File "/usr/lib/python2.7/site-packages/paramiko/pkey.py", line 196, in from_private_key_file
      key = cls(filename=filename, password=password)
    File "/usr/lib/python2.7/site-packages/paramiko/rsakey.py", line 45, in __init__
      self._from_private_key_file(filename, password)
    File "/usr/lib/python2.7/site-packages/paramiko/rsakey.py", line 163, in _from_private_key_file
      data = self._read_private_key_file('RSA', filename, password)
    File "/usr/lib/python2.7/site-packages/paramiko/pkey.py", line 268, in _read_private_key_file
      data = self._read_private_key(tag, f, password)
    File "/usr/lib/python2.7/site-packages/paramiko/pkey.py", line 277, in _read_private_key
      raise SSHException('not a valid ' + tag + ' private key file')
  SSHException: not a valid RSA private key file
  
• connecting to openssh server windows 10

  i have windows10 , i got a new task in my work to install openssh server and connect to it using putty. i installed openssh server on my pc (i added it from windows 10 features ) and its fine. now whats driving me crazy is whats username and passwrod to connect to it! i mean what i put here (cmd): ssh username@machineAdress .

  where can i find username and password please? i even tried to put username and password of my pc ... please i am so confused , where i find username and password thats is my problem guys help please.
• Can't log on to sftp server from a remote computer

  I am trying to set up a sftp server on my windows machine (Windows 7) and followed this tutorial https://winscp.net/eng/docs/guide_windows_openssh_server step-by-step.

  The server is running and I am able to connect to it via winscp on the machine I set up SFTP on, but I am unable to connect to it from any other computers.

  Some of the common errors I have gotten are: 1. Connection Timeout 2. Connection Refused 3. Password denied

  Different things I've tried: - For username, I have tried using the account name of the computer as well as the actual username of the machine found by "whoami" command. - For hostname, I used "nslookup" to find IP address, the ipV4 address, and also used default server name. - I ping'd all of the IP addresses and server address I've tried on remote machines and have gotten responses from all of them. - I even manually added the hostkey to known_hosts to make sure that the host address is found. However, none of the above worked. I've also set it up so that it takes either a password or a private key, so I didn't think there was an issue with that.

  I am very confused with this issue because I am able to connect to it on winscp on the machine I set up sftp on. I wonder what piece of information I am missing in my remote computers that I am not able to log on to it at all. (possibly somthing with network, hostname, public/private keys?)

  For username, I have tried using the account name of the computer as well as the actual username of the machine found by "whoami" command.

  For hostname, I used "nslookup" to find IP address, the ipV4 address, and also used default server name.

  I even manually added the hostkey to known_hosts to make sure that the host address is found.

  However, none of the above worked. I've also set it up so that it takes either a password or a private key, so I didn't think there was an issue with that.
• Why Are Some Characters Incapable of Being Pulled From Linux/Bash Environment Variables Into PHP?

  I'll try to make this as short as possible without making it completely unreadable.

  I'm storing encrypted data-sets in my database using PHP's openssl implementation.

  Here's something similar to what I'm doing:

  $pw = 'nobody loves you like I do.';
  $cm = 'aes-256-cbc-hmac-sha256';
  $c_key = '-$k���۱�d��o��ּ)2�"�C�!{���"�pO�⌧Ӝ�A��)mL�ڡ��i����OP753D8⮩eÒ98u07ף�11�u�e';
  $iv = '�w�[(32';
  $sslpw = openssl_encrypt($pw, $cm, $c_key, $options=0, $iv);
  

  The values I'm populating $c_key and $iv with were all generated using openssl_random_pseudo_bytes()

  So, thinking there'd be no issue here, I made a small file with all such values to be sourced into environment variables like so:

  export PAXYZ_C_KEY='-$k���۱�d��o��ּ)2�"�C�!{���"�pO�⌧Ӝ�A��)mL�ڡ��i����OP753D8⮩eÒ98u07ף�11�u�e';
  export PAXYZ_C_IV='�w�[(32'
  

  However, after I source the file, and read them into my PHP script, I lose a good 50% of each string even though they're string literals.

  However, if I set the env var directly via command line, it sets just fine. The issue is sticking it in a file and setting it within the file. I don't even know why this would make a difference, but it does.

  If I keep everything strictly PHP, there are no issues with any of these characters.

  At what level can I even diagnose this? It doesn't seem to matter whether I copy and paste into the terminal or an editor - the result is the same. But really, this is about where my expertise comes to a grinding halt.

  Unfortunately, I've already done much encrypting with my current set of keys - so I'm not too hot on re-encrypting everything if I don't have to.