středa 19. října 2016

jOpenSpace.cz 2016 - dobíječka baterek


Po loňské pauze se mi opět podařilo dostat na nekonferenci jOpenSpace, která probíhala o víkendu 7.-9.10. kousek od Pelhřimova. V průběhu víkendu měl každý z účastníků vyhrazených 10 minut aby osvětlil (lightning talk formou) své téma. Já jsem prezentoval projekt Sunstone, kterému jsem se věnoval v uplynulém roce v souvislosti s testováním JBoss EAP v cloudech a v Dockeru. Program konference na svém blogu shrnul Jarda Holaň, takže já zmíním jen krátce co mě zaujalo a jaký jsem měl z celého víkendu dojem.

Bezpečnost

Mile mě překvapilo, že několik prezentací bylo zaměřeno na bezpečnost a její testování. Já jsem si při nich do svého TODO-listu připsal položky:
  • projít si WebGoat (deliberately insecure web application maintained by OWASP)
  • vyzkoušet Burp Suite na penetrační testování

Headhunteři

V pátek při večeři probíhala zajímavá diskuse s klukama ze společnosti Zeebra Recruiting, kteří nám dali nahlédnout pod pokličku svého zaměstnání. Mluvilo se o tom, jak se vyhnout spamování od headhunterů, jak správně zacílit na oblast, do které by se chtěl člověk posunout. A velkým tématem byl i převis poptávky nad nabídkou vývojářů a důsledky tohoto stavu.

JavaScript

JavaScript prý ještě není mrtvý... ale žádná ze souvisejících přednášek mě nepřesvědčila o tom, že by stálo za to se mu nějak víc věnovat. Do svého Wish-listu jsem si připsal poznámku, že bych se mohl mrknout na TypeScript.

Enterprise Java

... v našich krajinách moc nežije. Změní se přístup s příchodem projektu WildFly Swarm, který na konferenci prezentoval Lukáš Vlček?

Performance

Zaujaly mě Feršovy milisekundy a jak se mění přístup k RESTu. Záznam stejné prezentace, ale s lepší časovou dotací najdete na Slideslive (CZJUG).

Dobíječka baterek

jOpenSpace je (alespoň u mě) velmi výkoná dobíječka baterek. Nejen že mě namotivovala, abych se podíval na nové tooly, jazyky a postupy a případně si aktualizoval CV-čko, ale hlavně to je konference, kde se každý baví s každým a networking tu opravdu funguje. Řeší se tu reálné problémy, bojují tu technologie a člověk ať chce či nechce vystoupí ze své bubliny (u mě Java EE) a nadechne se více-či-méně voňavého vzduchu reálného světa.

Byl jsem letos v Polsku na Devoxxu a s čistým svědomím můžu říct, že z jOpenSpace jsem si odnesl víc (a jestli se nepletu, tak i piva jsem tu vypil více).

Díky na konec

Díky patří všem organizátorům (Zdeněk Henek, Novoj, Jety, Petr Jůza) za nasazení a skvěle zvládnutou akci, sponzorům za sponzoring (seznam najdete na http://jopenspace.cz/) a všem účastníkům za atmosféru, přednášky, názory, ....

středa 7. září 2016

Clean-up blobs in Azure storage account with Azure CLI

If you realize at some point of your bright Azure future you don't remove VM disks correctly (like me in JCLOUDS-1170) you'll probably want to do a storage account clean up. Here is, how I did it on my linux desktop using Azure CLI.

As I wanted to keep some private OS images which were located in the storage account too, I moved them first into a newly created storage container.

# Prerequisities
  • installed azure-cli (I personally use the latest docker image microsoft/azure-cli with the tool)
  • installed jq tool for parsing JSON

# Set up

STORAGE_ACCOUNT="PutYourStorageAccountNameHere"
STORAGE_KEY="PutYourStorageAccountAccessKeyHere"

# Move/backup OS images to a new container

# create new storage container for OS image blobs
azure storage container create -a $STORAGE_ACCOUNT -k "$STORAGE_KEY" os-images

# linux images to be kept
for IMG in eap-7-rhel-7.2 eap-7-rhel-7.2-service; do
ORIG_BLOB=$(azure vm image show --json $IMG | jq -Mr '.mediaLinkUri')
echo "Original blob URL: $ORIG_BLOB"
# delete image, but keep the blob in storage account
azure vm image delete $IMG
# copy the blob to a new container and create image from it
azure vm image create --blob-url https://$STORAGE_ACCOUNT.blob.core.windows.net/os-images/$IMG.vhd --os linux --source-key "$STORAGE_KEY" $IMG $ORIG_BLOB
done

# Remove containers (with blobs)

# list container names (without the newly created one) to be removed
azure storage container list -a $STORAGE_ACCOUNT -k "$STORAGE_KEY" --json | jq -rM '.[] | .name' |grep -v os-images > /tmp/containers.txt

# for each container remove all blobs in it and then remove container
for CONTAINER in `cat /tmp/containers.txt`; do
azure storage blob list -a $STORAGE_ACCOUNT -k "$STORAGE_KEY" --json $CONTAINER | jq -rM '.[] | .name' > /tmp/vhds.txt
for BLOB in `cat /tmp/vhds.txt`; do
echo "Removing $BLOB"
# let's try to breake blob lease (if there exists one), otherwise we'll not be able to remove it
azure storage blob lease break -a $STORAGE_ACCOUNT -k "$STORAGE_KEY" $CONTAINER $BLOB
# remove blob
azure storage blob delete -q -a $STORAGE_ACCOUNT -k "$STORAGE_KEY" $CONTAINER $BLOB
done
# let's try to breake container lease (if there exists one), otherwise we'll not be able to remove it
azure storage container lease break -a $STORAGE_ACCOUNT -k "$STORAGE_KEY" $CONTAINER
# remove storage container
azure storage container delete -q -a $STORAGE_ACCOUNT -k "$STORAGE_KEY" $CONTAINER
done

neděle 21. srpna 2016

Create your own SSH-able Windows image with JBoss EAP on Azure

This entry describes one of possible ways how to create SSH-able Windows image with JBoss EAP 7 installed on MS Azure cloud (classic VM mode - asm). It's written mostly as commands with some comments - use your imagination (or a boring documentation), when you're not sure what does it do.

If you use these step, then do it carefully - it disables the firewall for instance! Don't forget to change the USER_PASSWORD value!

# Prerequisites

# Azure CLI - install and configure classic mode

# install either Azure CLI for your system or use Docker image provided by Microsoft


# login and configure classic mode
azure login
azure config mode asm

# Create Azure storage account (with container for the image)

# Create VM from public Windows image

# name of virtual machine used to prepare new OS image
export AZURE_HOST=eap7-prepare
# target image name
export IMGNAME=eap-7-win-2012r2
# export IMGNAME=eap-7-win-2012r2-service
# there will be a 'jboss' user created and this will be its password
# CHANGE ME!!
export USER_PASSWORD=Q8EnPU3.r9


azure vm create $AZURE_HOST a699494373c04fc0bc8f2bb1389d6106__Windows-Server-2012-R2-20160617-en.us-127GB.vhd --location "East US" --vm-size Basic_A1 -r -u https://eap7server4548.blob.core.windows.net/vhds/${AZURE_HOST}.vhd jboss "${USER_PASSWORD}"
azure vm endpoint create $AZURE_HOST 22; azure vm endpoint create $AZURE_HOST 3389

# Fix rdesktop connection from outside the Azure (CredSSP required by server)

# connect (mstsc) from another windows machine on Azure and fix configuration
reg add "HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Terminal Server\WinStations\RDP-Tcp" /v UserAuthentication /t REG_DWORD /d 0 /f

# Reconfigure Firewall - set permitting policy

# check current configuration
netsh advfirewall show allprofiles
# set policy to allow-all
netsh advfirewall set allprofiles firewallpolicy allowinbound,allowoutbound
# disable firewall for all profiles
netsh advfirewall set allprofiles state off

# Install software

# connect with rdesktop
rdesktop -k en-us -g 1600x800 -r clipboard:PRIMARYCLIPBOARD -u jboss -p "${USER_PASSWORD}" ${AZURE_HOST}.cloudapp.net


# Software installation steps do as Administrator
# Start -> type "cmd" -> right click -> "Run as Administrator"
# or "runas /user:administrator cmd.exe"


mkdir C:\install
cd \install

# Cygwin with SSH server

# Download installer from command line (we don't have wget or curl yet, so let's use bitsadmin)
bitsadmin  /transfer downloadcygwin  /download  /priority normal https://www.cygwin.com/setup-x86_64.exe  C:\install\setup-x86_64.exe


setup-x86_64.exe --quiet-mode --packages=bash,vim,openssh,bash-completion,lynx,wget,curl,git,diffutils,patchutils,python,python3,less,unzip --site http://mirrors.kernel.org/sourceware/cygwin/ --root C:\cygwin

# Regenerate /etc/passwd when configuring SSH server

# We have to call somehow cygwins 'mkpasswd -l -b >/etc/passwd' during boot and generate new passwd file because during provisioning the host identification changes


# open the Cygwin terminal - again "Run as Administrator"
# fix the patch - add empty-line at the end
echo >> ssh-host-config.patch
# apply patch
patch /usr/bin/ssh-host-config ssh-host-config.patch
# run the patched SSH configuration script (new account with random password will be created for the service)
ssh-host-config -y -c ntsec -u sshd_account -w `openssl rand -base64 12`


# Reboot the machine now

# Install IIS (if needed)

DISM.EXE /enable-feature /online /featureName:IIS-WebServerRole /featureName:IIS-WebServer
/featureName:IIS-CommonHttpFeatures /featureName:IIS-StaticContent /featureName:IIS-DefaultDocument
/featureName:IIS-DirectoryBrowsing /featureName:IIS-HttpErrors /featureName:IIS-HttpRedirect
/featureName:IIS-ApplicationDevelopment /featureName:IIS-ISAPIExtensions /featureName:IIS-ISAPIFilter /featureName:IIS-HealthAndDiagnostics /featureName:IIS-HttpLogging /featureName:IIS-LoggingLibraries
/featureName:IIS-Security /featureName:IIS-RequestFiltering /featureName:IIS-Performance /featureName:IIS-HttpCompressionStatic /featureName:IIS-WebServerManagementTools
/featureName:IIS-WebSockets /featureName:IIS-ManagementConsole


# dism /online /getFeatures /format:table
# dism /online /getFeatureinfo /featureName:iis-asp
# dism /online /get-packages /format:table
# dism /online /getpackageinfo /packagename: Package_for_KB2880289~31bf3856ad364e35~amd64~~6.3.1.0
# dism /online /enable-feature /featureName:iis-asp [/source]
# dism /online /disable-feature /featureName:iis-asp [/remove]

# Install Java (Cygwin)

wget -c --no-check-certificate --no-cookies --header "Cookie: oraclelicense=accept-securebackup-cookie" "http://download.oracle.com/otn-pub/java/jdk/8u91-b14/jdk-8u91-windows-x64.exe"
chmod +x jdk-*.exe
jdk-*.exe /s 'ADDLOCAL=ToolsFeature,SourceFeature,PublicjreFeature' 'INSTALLDIR=C:\Java\jdk1.8'
# configure PATH property
setx PATH "%PATH%;C:\Java\jdk1.8\bin"
# configure JAVA_HOME (machine level)
setx /M JAVA_HOME 'C:\Java\jdk1.8'

# Install EAP from ZIP (Cygwin)

# copy EAP and JSVC to the running VM (or you can use download from customer portal)
scp jboss-eap-7.0.0.zip jbcs-jsvc-1.0.15-win6.x86_64.zip jboss@$AZURE_HOST.cloudapp.net:/cygdrive/c/install


# after SSH to VM do
mkdir /cygdrive/c/eap
cd /cygdrive/c/eap
unzip -q /cygdrive/c/install/jboss-eap-7.0.0.zip
unzip -q /cygdrive/c/install/jbcs-jsvc-1.0.15-win6.x86_64.zip


# bind to all interfaces
echo -ne 'set "JAVA_OPTS=%JAVA_OPTS% -Djboss.bind.address=0.0.0.0 -Djboss.bind.address.management=0.0.0.0"\r\n' >> /cygdrive/c/eap/jboss-eap-7.0/bin/standalone.conf.bat
# add a Management user
/cygdrive/c/eap/jboss-eap-7.0/bin/add-user.sh -u eapqe -p "${USER_PASSWORD}" -s

Install ISAPI redirector to the IIS (if needed)


# grant permissions for IIS users - expecting the isapi_redirector in C:\connectors\
cmd /c icacls 'C:\connectors' /grant 'IIS_IUSRS:F'
cmd /c 'C:\Windows\system32\inetsrv\appcmd.exe' set config /section:isapiCgiRestriction "/+[path='C:\connectors\isapi_redirect.dll',description='jboss',allowed='True']"
cmd /c 'C:\Windows\system32\inetsrv\appcmd.exe' add vdir '/app.name:Default Web Site/' '/path:/jboss' '/physicalPath:C:\connectors\'
cmd /c 'C:\Windows\system32\inetsrv\appcmd.exe' unlock config /section:isapiFilters
cmd /c 'C:\Windows\system32\inetsrv\appcmd.exe' set config 'Default Web Site' /section:isapiFilters "/+[name='jboss',path='C:\connectors\isapi_redirect.dll',enabled='True']"
cmd /c 'C:\Windows\system32\inetsrv\appcmd.exe' set config /section:handlers /accessPolicy:Read,Script,Execute

# Register EAP as a service

cd /cygdrive/c/eap/jboss-eap-7.0/bin
chmod +x service.bat
./service.bat install /startup /config standalone-full-ha.xml

# Deprovision and Create image from the VM

# It seems the amount of "deprovisionings" is limited - you can hit Fatal error when using the sysprep tool


# generalize the VM, so the OS image can be created from it
cd /cygdrive/c/Windows/System32/Sysprep
./sysprep.exe /oobe /generalize /shutdown /quiet


azure vm shutdown $AZURE_HOST
# create the new image
azure vm capture -t $AZURE_HOST $IMGNAME

# Start VM from the image

azure vm create eap7-test $IMGNAME --location "East US" --ssh 22 --vm-size Basic_A1 jboss "${USER_PASSWORD}"

# Other / links

Storage management:
About disks:
How to handle custom-data?

pondělí 8. srpna 2016

Ubuntu (Mate 16.04) - disable WiFi when ethernet cable is plugged in

Use following script to create rule, which disables wifi if the ethernet cable connection is up. The solution is based on this stackoverflow article.
sudo cat > /etc/NetworkManager/dispatcher.d//99-disable-wifi-if-not-needed <<EOT
#!/bin/bash

if [ "$1" = "enp0s25" ]; then
    case "$2" in
        up)
            nmcli radio wifi off
            ;;
        down)
            nmcli radio wifi on
            ;;
    esac
fi

Replace the enp0s25 with the correct "cable" interface if needed.

pondělí 25. dubna 2016

Ignore the boring SSH error message - Host identification has changed!

The problem

If you work with virtual machines in clouds, or you run an SSH server in Docker containers, then you've probably met the following error message during making ssh connection:
(I'm connecting through SSH to a docker container)
~$ ssh -p 8822 root@localhost
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.
The fingerprint for the ECDSA key sent by the remote host is
SHA256:smYv5yA0n9/YrBgJMUCk5dYPWGj7bTpU40M9aFBQ72Y.
Please contact your system administrator.
Add correct host key in /home/jcacek/.ssh/known_hosts to get rid of this message.
Offending ECDSA key in /home/jcacek/.ssh/known_hosts:107
  remove with:
  ssh-keygen -f "/home/jcacek/.ssh/known_hosts" -R [localhost]:8822
ECDSA host key for [localhost]:8822 has changed and you have requested strict checking.
Host key verification failed.
And as a result the ssh refused to connect to requested server.

The problem is, you are reusing the host/port combination which was already registered in your system and the SSH client tries to keep you on a safe side. It doesn't connect to a server whose public key doesn't match the one registered in your system.

The obvious solution

Yes, I know. You say, the message suggest me a solution. Just to run
ssh-keygen -f "/home/jcacek/.ssh/known_hosts" -R [localhost]:8822
... and everything works correctly. Or doesn't it? Let's try.
~$ ssh -p 8822 root@localhost
The authenticity of host '[localhost]:8822 ([127.0.0.1]:8822)' can't be established.
ECDSA key fingerprint is SHA256:smYv5yA0n9/YrBgJMUCk5dYPWGj7bTpU40M9aFBQ72Y.
Are you sure you want to continue connecting (yes/no)?
Oh My Java! I have to write "yes". And No! the "y", "Y", "yep" etc. don't work.
Are you sure you want to continue connecting (yes/no)? y
Please type 'yes' or 'no': Y
Please type 'yes' or 'no': yep
Please type 'yes' or 'no': I give up!
Please type 'yes' or 'no': 
So it's really irritating and it takes time to handle it.

The real solution

So what can we do with it? Just add some more arguments to our ssh command
ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -p 8822 root@localhost
By using the UserKnownHostsFile option we said we don't want to use ~/.ssh/known_host file, but rather the one provided as a value of this option. And the /dev/null is always empty (i.e. it can't cause a conflict with the checked server key).

The "no" value in StrictHostKeyChecking option disables the question if the new key can be stored into the provided known hosts file.

To make it simpler, just add an alias into your system. E.g. this is a line in my ~/.bash_aliases file:
alias sshx='ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null'

Warning

Use this only if you are aware of risks coming from the relaxing SSH security mechanisms.

pondělí 15. února 2016

Playing live demos on Linux

I like live demos during presentations. When they work and don't take too much time.

And I don't like when the presenter types too much.

So I've created the demo-step script.

History

I've tried to find a way how to "play" live demos (on Linux) in a natural way. First I've found solution by putting keyboard shortcuts into the ~/.inputrc configuration file (example here). The disadvantage is, you have to remember, what's the next shortcut.

I'm not so good in remembering shortcuts, so I've created simpler solution by utilizing xdotool Linux application (xdotool home).

Sample usage

You can find sample presentation with uses the demo-step on my GitHub.

You just need to put the live demo commands into the ~/demo.commands text file and then play them by running demo-step after pressing a keyboard shortcut.

Sample content of the ~/demo.commands:

#\n# search images in Docker public registry (Docker hub)\ndocker search wildfly
#\n# run container from an image\ndocker run -it jboss/wildfly

And the output in the console window after the first demo-step script execution:

$ #
$ # search images in Docker public registry (Docker hub)
$ docker search wildfly