Do I Need to Learn Linux and R and Python?

Question

Do I Need to Learn Linux and R and Python?

Steve Jones - SSC Editor

SSC Guru

Points: 736078
More actions
May 25, 2017 at 8:29 pm

#343020

Comments posted to this topic are about the item Do I Need to Learn Linux and R and Python?

Viewing 10 posts - 1 through 9 (of 9 total)

You must be logged in to reply to this topic. Login to reply

Robert Sterbal SSChampion Points: 11049 More actions · Answer 1

For other reasons (my company's IT group blocks Apache on Windows) I wound up having to install Linux in a Virtualbox on my desktop.

I could use some help with BASH on the Linux side to automate the transfer of my backup from the VM to my desktop. I'm looking forward to people's suggestions!

412-977-3526 call/text

Jeff Moden SSC Guru Points: 1004290 More actions · Answer 2

Heh... I absolutely agree that learning new things is essential in this industry. For many of the DBAs and Developers that I've interviewed over the last decade, I would suggest that "something new" be <drum roll please> T-SQL. 😉 <Ba-da-boom-tish!>

--Jeff Moden

RBAR is pronounced "ree-bar" and is a "Modenism" for Row-By-Agonizing-Row.
First step towards the paradigm shift of writing Set Based code:
________Stop thinking about what you want to do to a ROW... think, instead, of what you want to do to a COLUMN.

Change is inevitable... Change for the better is not.

Helpful Links:
How to post code problems
How to Post Performance Problems
Create a Tally Function (fnTally)

Revenant SSC-Forever Points: 42467 More actions · Answer 3

IMO question is not whether but to what depth. There are some 68k lPython ibraries (!). If you wanted to take 5 minutes for each just to find what it can do for you would be 5 years full time, provided that you have absolute memory.

R is a data science language with "only" hundreds of libraries but assumes that you have expert understanding of statistics and matrix calculus.

IMO an SQL Server generalist should learn to make data scientists' code run and help them with debugging.

x SSC-Insane Points: 23660 More actions · Answer 4

robert.sterbal 56890 - Friday, May 26, 2017 8:53 AM
For other reasons (my company's IT group blocks Apache on Windows) I wound up having to install Linux in a Virtualbox on my desktop.
I could use some help with BASH on the Linux side to automate the transfer of my backup from the VM to my desktop. I'm looking forward to people's suggestions!

I would use putty's pscp to copy the backup files from Linux to windows, and then schedule that in a batch file with windows scheduler. You can produce a key to better automate the ssh connection that removes the password requirement (because the ssh key is stored in the registry). The reason I would take this direction is that connecting to Linux seems to have more options than connecting to windows, not that its impossible or anything, but ssh is just so easy to work with.

Robert Sterbal SSChampion Points: 11049 More actions · Answer 5

Here is the DOS script I created:

REM set the name of the root backup folder with no trailing \
SET backupfolder=C:\apps\bitnami-vm\backups-cl

SET dirname="%backupfolder%\%date:~10,4%-%date:~4,2%-%date:~7,2%_%time:~0,2%-%time:~3,2%-%time:~6,2%"
echo %dirname%
md %dirname%
"c:\Program Files (x86)\PuTTY\pscp.exe" -pw ?PasswordGoesHere? bitnami@ubuntu.robot-rx.com:/home/bitnami/backups/* %dirname%
pause
exit

Is it possible to call this batch file, or execute my BASH script:

#!/bin/bash
PWD=`cat /home/bitnami/backups/pwd.cfg`
#echo x$PWD x
DATETIME=`date +%Y-%m-%d-%H-%M-%S`
mkdir -p /home/bitnami/tmp/
mkdir -p /home/bitnami/tmp/backup1/
mkdir -p /home/bitnami/tmp/backup2/
mkdir -p /home/bitnami/tmp/backup3/
rm /home/bitnami/tmp/backup3/*
mv /home/bitnami/tmp/backup2/Robert-s-wiki-* /home/bitnami/tmp/backup3/
mv /home/bitnami/tmp/backup1/Robert-s-wiki-* /home/bitnami/tmp/backup2/
mv /home/bitnami/backups/Robert-s-wiki-* /home/bitnami/tmp/backup1/
#cp /home/bitnami/backups/Robert-s-wiki-* /home/bitnami/tmp/backup1/
#rm /home/bitnami/backups/Robert-s-wiki-*
mysqldump -u root -p$PWD bitnami_mediawiki > /home/bitnami/backups/Robert-s-wiki-$DATETIME.sql
cp /home/bitnami/apps/mediawiki/htdocs/resources/assets/wiki.png /home/bitnami/backups/wiki.png
cp /opt/bitnami/apps/mediawiki/htdocs/LocalSettings.php /home/bitnami/backups/LocalSettings.php
cp /home/bitnami/mw-backup.sh /home/bitnami/backups/mw-backup.sh
cp /opt/bitnami/mysql/my.cnf /home/bitnami/backups/my.cnf
echo files in the backups folder:
ls -l /home/bitnami/backups/
echo current backup:
ls -l /home/bitnami/backups/Robert-s-wiki-$DATETIME.sql

with one click?

412-977-3526 call/text

x SSC-Insane Points: 23660 More actions · Answer 6

robert.sterbal 56890 - Thursday, June 1, 2017 11:30 AM
Here is the DOS script I created:
REM set the name of the root backup folder with no trailing \
SET backupfolder=C:\apps\bitnami-vm\backups-cl
SET dirname="%backupfolder%\%date:~10,4%-%date:~4,2%-%date:~7,2%_%time:~0,2%-%time:~3,2%-%time:~6,2%"
echo %dirname%
md %dirname%
"c:\Program Files (x86)\PuTTY\pscp.exe" -pw ?PasswordGoesHere? bitnami@ubuntu.robot-rx.com:/home/bitnami/backups/* %dirname%
pause
exit
Is it possible to call this batch file, or execute my BASH script:

#!/bin/bash
PWD=`cat /home/bitnami/backups/pwd.cfg`
#echo x$PWD x
DATETIME=`date +%Y-%m-%d-%H-%M-%S`
mkdir -p /home/bitnami/tmp/
mkdir -p /home/bitnami/tmp/backup1/
mkdir -p /home/bitnami/tmp/backup2/
mkdir -p /home/bitnami/tmp/backup3/
rm /home/bitnami/tmp/backup3/*
mv /home/bitnami/tmp/backup2/Robert-s-wiki-* /home/bitnami/tmp/backup3/
mv /home/bitnami/tmp/backup1/Robert-s-wiki-* /home/bitnami/tmp/backup2/
mv /home/bitnami/backups/Robert-s-wiki-* /home/bitnami/tmp/backup1/
#cp /home/bitnami/backups/Robert-s-wiki-* /home/bitnami/tmp/backup1/
#rm /home/bitnami/backups/Robert-s-wiki-*
mysqldump -u root -p$PWD bitnami_mediawiki > /home/bitnami/backups/Robert-s-wiki-$DATETIME.sql
cp /home/bitnami/apps/mediawiki/htdocs/resources/assets/wiki.png /home/bitnami/backups/wiki.png
cp /opt/bitnami/apps/mediawiki/htdocs/LocalSettings.php /home/bitnami/backups/LocalSettings.php
cp /home/bitnami/mw-backup.sh /home/bitnami/backups/mw-backup.sh
cp /opt/bitnami/mysql/my.cnf /home/bitnami/backups/my.cnf
echo files in the backups folder:
ls -l /home/bitnami/backups/
echo current backup:
ls -l /home/bitnami/backups/Robert-s-wiki-$DATETIME.sql

with one click?

What I used for that was "plink", its part of the putty download if you download the zip. SSH lets you run shell commands remotely, and plink lets you do this from your windows host using the SSH protocol. You could put a "plink" line in your windows batch file that calls the Linux shell script you created. The "plink" line would run the Linux shell script, and then the next line in your batch file would then copy the files to your windows host with pscp which looks like you got set up already. This way the whole job runs from your windows batch file with no extra clicks needed, obviously you can make two batch files too.

http://the.earth.li/~sgtatham/putty/0.53b/htmldoc/Chapter7.html gives you the lowdown on how to use plink with the parameters that you'd need.

Robert Sterbal SSChampion Points: 11049 More actions · Answer 7

THANK YOU SO MUCH!!!!

This is my one click backup script:

REM Generate the backup

"C:\Program Files (x86)\PuTTY\plink.exe" -pw ?password? bitnami@ubuntu.robot-rx.com bash mw-backup.sh

REM set the name of the root backup folder with no trailing \

SET backupfolder=C:\apps\bitnami-vm\backups-cl

SET dirname="%backupfolder%\%date:~10,4%-%date:~4,2%-%date:~7,2%_%time:~0,2%-%time:~3,2%-%time:~6,2%"
echo %dirname%
md %dirname%
"c:\Program Files (x86)\PuTTY\pscp.exe" -pw ?password? bitnami@ubuntu.robot-rx.com:/home/bitnami/backups/* %dirname%
pause
exit

412-977-3526 call/text

Lou SSCrazy Points: 2929 More actions · Answer 8

Hi,
Sometimes, we need to leave our comfort zone and learn some new tricks.
Python. R are small changes for Microsoft, but big changes to our community which surely will bring new opportunities.
Best regards

Revenant SSC-Forever Points: 42467 More actions · Answer 9

luismarinaray - Saturday, June 3, 2017 8:06 PM
Hi,
Sometimes, we need to leave our comfort zone and learn some new tricks.
Python. R are small changes for Microsoft, but big changes to our community which surely will bring new opportunities.
Best regards

Python is not a small change for Microsoft: it has quickly become a first class citizen and is set to be THE language of Azure. Note, for example, the quick adoption of Jupyter Notebooks and prominence of Python modules in Machine Learning Studio. I could continue but it would get us far from SQL Server.