středa 7. září 2016

Clean-up blobs in Azure storage account with Azure CLI

If you realize at some point of your bright Azure future you don't remove VM disks correctly (like me in JCLOUDS-1170) you'll probably want to do a storage account clean up. Here is, how I did it on my linux desktop using Azure CLI.

As I wanted to keep some private OS images which were located in the storage account too, I moved them first into a newly created storage container.

# Prerequisities
  • installed azure-cli (I personally use the latest docker image microsoft/azure-cli with the tool)
  • installed jq tool for parsing JSON

# Set up

STORAGE_ACCOUNT="PutYourStorageAccountNameHere"
STORAGE_KEY="PutYourStorageAccountAccessKeyHere"

# Move/backup OS images to a new container

# create new storage container for OS image blobs
azure storage container create -a $STORAGE_ACCOUNT -k "$STORAGE_KEY" os-images

# linux images to be kept
for IMG in eap-7-rhel-7.2 eap-7-rhel-7.2-service; do
ORIG_BLOB=$(azure vm image show --json $IMG | jq -Mr '.mediaLinkUri')
echo "Original blob URL: $ORIG_BLOB"
# delete image, but keep the blob in storage account
azure vm image delete $IMG
# copy the blob to a new container and create image from it
azure vm image create --blob-url https://$STORAGE_ACCOUNT.blob.core.windows.net/os-images/$IMG.vhd --os linux --source-key "$STORAGE_KEY" $IMG $ORIG_BLOB
done

# Remove containers (with blobs)

# list container names (without the newly created one) to be removed
azure storage container list -a $STORAGE_ACCOUNT -k "$STORAGE_KEY" --json | jq -rM '.[] | .name' |grep -v os-images > /tmp/containers.txt

# for each container remove all blobs in it and then remove container
for CONTAINER in `cat /tmp/containers.txt`; do
azure storage blob list -a $STORAGE_ACCOUNT -k "$STORAGE_KEY" --json $CONTAINER | jq -rM '.[] | .name' > /tmp/vhds.txt
for BLOB in `cat /tmp/vhds.txt`; do
echo "Removing $BLOB"
# let's try to breake blob lease (if there exists one), otherwise we'll not be able to remove it
azure storage blob lease break -a $STORAGE_ACCOUNT -k "$STORAGE_KEY" $CONTAINER $BLOB
# remove blob
azure storage blob delete -q -a $STORAGE_ACCOUNT -k "$STORAGE_KEY" $CONTAINER $BLOB
done
# let's try to breake container lease (if there exists one), otherwise we'll not be able to remove it
azure storage container lease break -a $STORAGE_ACCOUNT -k "$STORAGE_KEY" $CONTAINER
# remove storage container
azure storage container delete -q -a $STORAGE_ACCOUNT -k "$STORAGE_KEY" $CONTAINER
done

neděle 21. srpna 2016

Create your own SSH-able Windows image with JBoss EAP on Azure

This entry describes one of possible ways how to create SSH-able Windows image with JBoss EAP 7 installed on MS Azure cloud (classic VM mode - asm). It's written mostly as commands with some comments - use your imagination (or a boring documentation), when you're not sure what does it do.

If you use these step, then do it carefully - it disables the firewall for instance! Don't forget to change the USER_PASSWORD value!

# Prerequisites

# Azure CLI - install and configure classic mode

# install either Azure CLI for your system or use Docker image provided by Microsoft


# login and configure classic mode
azure login
azure config mode asm

# Create Azure storage account (with container for the image)

# Create VM from public Windows image

# name of virtual machine used to prepare new OS image
export AZURE_HOST=eap7-prepare
# target image name
export IMGNAME=eap-7-win-2012r2
# export IMGNAME=eap-7-win-2012r2-service
# there will be a 'jboss' user created and this will be its password
# CHANGE ME!!
export USER_PASSWORD=Q8EnPU3.r9


azure vm create $AZURE_HOST a699494373c04fc0bc8f2bb1389d6106__Windows-Server-2012-R2-20160617-en.us-127GB.vhd --location "East US" --vm-size Basic_A1 -r -u https://eap7server4548.blob.core.windows.net/vhds/${AZURE_HOST}.vhd jboss "${USER_PASSWORD}"
azure vm endpoint create $AZURE_HOST 22; azure vm endpoint create $AZURE_HOST 3389

# Fix rdesktop connection from outside the Azure (CredSSP required by server)

# connect (mstsc) from another windows machine on Azure and fix configuration
reg add "HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Terminal Server\WinStations\RDP-Tcp" /v UserAuthentication /t REG_DWORD /d 0 /f

# Reconfigure Firewall - set permitting policy

# check current configuration
netsh advfirewall show allprofiles
# set policy to allow-all
netsh advfirewall set allprofiles firewallpolicy allowinbound,allowoutbound
# disable firewall for all profiles
netsh advfirewall set allprofiles state off

# Install software

# connect with rdesktop
rdesktop -k en-us -g 1600x800 -r clipboard:PRIMARYCLIPBOARD -u jboss -p "${USER_PASSWORD}" ${AZURE_HOST}.cloudapp.net


# Software installation steps do as Administrator
# Start -> type "cmd" -> right click -> "Run as Administrator"
# or "runas /user:administrator cmd.exe"


mkdir C:\install
cd \install

# Cygwin with SSH server

# Download installer from command line (we don't have wget or curl yet, so let's use bitsadmin)
bitsadmin  /transfer downloadcygwin  /download  /priority normal https://www.cygwin.com/setup-x86_64.exe  C:\install\setup-x86_64.exe


setup-x86_64.exe --quiet-mode --packages=bash,vim,openssh,bash-completion,lynx,wget,curl,git,diffutils,patchutils,python,python3,less,unzip --site http://mirrors.kernel.org/sourceware/cygwin/ --root C:\cygwin

# Regenerate /etc/passwd when configuring SSH server

# We have to call somehow cygwins 'mkpasswd -l -b >/etc/passwd' during boot and generate new passwd file because during provisioning the host identification changes


# open the Cygwin terminal - again "Run as Administrator"
# fix the patch - add empty-line at the end
echo >> ssh-host-config.patch
# apply patch
patch /usr/bin/ssh-host-config ssh-host-config.patch
# run the patched SSH configuration script (new account with random password will be created for the service)
ssh-host-config -y -c ntsec -u sshd_account -w `openssl rand -base64 12`


# Reboot the machine now

# Install IIS (if needed)

DISM.EXE /enable-feature /online /featureName:IIS-WebServerRole /featureName:IIS-WebServer
/featureName:IIS-CommonHttpFeatures /featureName:IIS-StaticContent /featureName:IIS-DefaultDocument
/featureName:IIS-DirectoryBrowsing /featureName:IIS-HttpErrors /featureName:IIS-HttpRedirect
/featureName:IIS-ApplicationDevelopment /featureName:IIS-ISAPIExtensions /featureName:IIS-ISAPIFilter /featureName:IIS-HealthAndDiagnostics /featureName:IIS-HttpLogging /featureName:IIS-LoggingLibraries
/featureName:IIS-Security /featureName:IIS-RequestFiltering /featureName:IIS-Performance /featureName:IIS-HttpCompressionStatic /featureName:IIS-WebServerManagementTools
/featureName:IIS-WebSockets /featureName:IIS-ManagementConsole


# dism /online /getFeatures /format:table
# dism /online /getFeatureinfo /featureName:iis-asp
# dism /online /get-packages /format:table
# dism /online /getpackageinfo /packagename: Package_for_KB2880289~31bf3856ad364e35~amd64~~6.3.1.0
# dism /online /enable-feature /featureName:iis-asp [/source]
# dism /online /disable-feature /featureName:iis-asp [/remove]

# Install Java (Cygwin)

wget -c --no-check-certificate --no-cookies --header "Cookie: oraclelicense=accept-securebackup-cookie" "http://download.oracle.com/otn-pub/java/jdk/8u91-b14/jdk-8u91-windows-x64.exe"
chmod +x jdk-*.exe
jdk-*.exe /s 'ADDLOCAL=ToolsFeature,SourceFeature,PublicjreFeature' 'INSTALLDIR=C:\Java\jdk1.8'
# configure PATH property
setx PATH "%PATH%;C:\Java\jdk1.8\bin"
# configure JAVA_HOME (machine level)
setx /M JAVA_HOME 'C:\Java\jdk1.8'

# Install EAP from ZIP (Cygwin)

# copy EAP and JSVC to the running VM (or you can use download from customer portal)
scp jboss-eap-7.0.0.zip jbcs-jsvc-1.0.15-win6.x86_64.zip jboss@$AZURE_HOST.cloudapp.net:/cygdrive/c/install


# after SSH to VM do
mkdir /cygdrive/c/eap
cd /cygdrive/c/eap
unzip -q /cygdrive/c/install/jboss-eap-7.0.0.zip
unzip -q /cygdrive/c/install/jbcs-jsvc-1.0.15-win6.x86_64.zip


# bind to all interfaces
echo -ne 'set "JAVA_OPTS=%JAVA_OPTS% -Djboss.bind.address=0.0.0.0 -Djboss.bind.address.management=0.0.0.0"\r\n' >> /cygdrive/c/eap/jboss-eap-7.0/bin/standalone.conf.bat
# add a Management user
/cygdrive/c/eap/jboss-eap-7.0/bin/add-user.sh -u eapqe -p "${USER_PASSWORD}" -s

Install ISAPI redirector to the IIS (if needed)


# grant permissions for IIS users - expecting the isapi_redirector in C:\connectors\
cmd /c icacls 'C:\connectors' /grant 'IIS_IUSRS:F'
cmd /c 'C:\Windows\system32\inetsrv\appcmd.exe' set config /section:isapiCgiRestriction "/+[path='C:\connectors\isapi_redirect.dll',description='jboss',allowed='True']"
cmd /c 'C:\Windows\system32\inetsrv\appcmd.exe' add vdir '/app.name:Default Web Site/' '/path:/jboss' '/physicalPath:C:\connectors\'
cmd /c 'C:\Windows\system32\inetsrv\appcmd.exe' unlock config /section:isapiFilters
cmd /c 'C:\Windows\system32\inetsrv\appcmd.exe' set config 'Default Web Site' /section:isapiFilters "/+[name='jboss',path='C:\connectors\isapi_redirect.dll',enabled='True']"
cmd /c 'C:\Windows\system32\inetsrv\appcmd.exe' set config /section:handlers /accessPolicy:Read,Script,Execute

# Register EAP as a service

cd /cygdrive/c/eap/jboss-eap-7.0/bin
chmod +x service.bat
./service.bat install /startup /config standalone-full-ha.xml

# Deprovision and Create image from the VM

# It seems the amount of "deprovisionings" is limited - you can hit Fatal error when using the sysprep tool


# generalize the VM, so the OS image can be created from it
cd /cygdrive/c/Windows/System32/Sysprep
./sysprep.exe /oobe /generalize /shutdown /quiet


azure vm shutdown $AZURE_HOST
# create the new image
azure vm capture -t $AZURE_HOST $IMGNAME

# Start VM from the image

azure vm create eap7-test $IMGNAME --location "East US" --ssh 22 --vm-size Basic_A1 jboss "${USER_PASSWORD}"

# Other / links

Storage management:
About disks:
How to handle custom-data?

pondělí 8. srpna 2016

Ubuntu (Mate 16.04) - disable WiFi when ethernet cable is plugged in

Use following script to create rule, which disables wifi if the ethernet cable connection is up. The solution is based on this stackoverflow article.
sudo cat > /etc/NetworkManager/dispatcher.d//99-disable-wifi-if-not-needed <<EOT
#!/bin/bash

if [ "$1" = "enp0s25" ]; then
    case "$2" in
        up)
            nmcli radio wifi off
            ;;
        down)
            nmcli radio wifi on
            ;;
    esac
fi

Replace the enp0s25 with the correct "cable" interface if needed.