TCP multiplexing for normal people

TCP is a connection oriented protocol, meaning that the protocol guarantees the order of information delivery in the same way a telephone connection ensures that the information in transmitted in correct order as compared to UDP where order and delivery are not guaranteed. (similar to sending postcards)

Establishing a connection

A three way handshake is used to establish a connection as follows

 

Once a connection is made then information can be transferred

Similarly several packets are required to tear down a connection as follows

As can be seen each connect has an overhead of 7 packets each with associated network latency.

Consider the bad old days of http 1.0. Every resource on a web page required a separate connection so multiple connection setups and tear downs slowed page loading. Also each connection would require memory and an entry in the session table reducing the scalability of the server

One of the major improvements in http 1.1 was to allow persistent connections

A persistent connection is one where the connection is not torn down once the client has received a response from the host. A webpage being served over http 1.1 would use a single connection for all resources both text and images.

Another example of this optimisation is with the SMPP (Short Message Peer to Peer) protocol. This is a protocol running over IP allowing hosts to transmit SMS messages to a telecommunications SMS controller. An example might be a bank which sends SMS messages as a client progresses through a loan application. This protocol uses the concept of “Binds” which are long lived TCP connections. An SMPP client will send an open bind command to the SMS controller and then send SMS messages ad hoc also sending keep alive packets to maintain the connection. Using this strategy a single connection can transmit thousands of sms messages.

TCP Multiplexing takes this further. In the previous examples each IP source IP would need a separate connection to the server to be able to transfer information. In home networking a single public IP address is used to provide internet access to a host of home devices using network address translation (NAT)

In the above example the NAT router uses a single public IP address 14.1.23.5 to serve 4 internal PCs. This is done by the router maintaining a natp table

A load balancer can use a similar strategy to allow many clients to share a single TCP connection on a webserver

The load balance will then terminate the connections from the client , translate the source port and IP address and multiplex requests down a single connection. This frees the webserver of the overhead of continual TCP connection and teardown, allowing a large number of clients to be serviced by a single server

Capturing traffic from Genymotion or any other virtual box based emulator without a proxy!

 

I was asked how one could capture traffic for Android applications that ignore proxy settings such as WhatsApp! Fortunately using the Genymotion or Xamarin emulator make it easy by doing a little virtual network plumbing. J

In summary the steps are as follows

  1. Create a new host only virtual box network adaptor
  2. Attach this new adaptor to the 2nd (WiFI ) interface of the emulator
  3. Bridge the new host adaptor with your internet connection
  4. Run wireshark and capture the traffic

Create a new “Host Only” adaptor.

Run virtualbox and go to File -> Preferences, this will bring up the settings menu

Click on the add icon and create a new Host Only Ethernet Adaptor

Now attach this new interface to the virtual machine in question

Click OK and save

Now we need to create the bridge between the Host Only virtual adaptor and the real Ethernet adaptor (or the WIFI if you want)

 

Go to adaptor settings in windows

Select the two adaptors to be bridged, use CTRL right click to individually select

Right Click on one of the selected adaptor s then select bridge connections

 

Let this run for a minute at which point a bridge adaptor will appear

 

 

You are now all set to capture

 

Run the emulator and track with a Wireshark capture on the physical adaptor

 

 

Have fun!

Reverse Engineering a SOAP web service in .NET (WCF)

As much as we complain about SOAP web services. SOAP has the great advantage in that everything we need to know to know is contained in the WSDL file. Also if you are interfacing to SAP and other ERP systems you can be sure that they will have a SOAP implementation

WDSL stands for (Web Service Definition Language) and provides us all the information we need, especially our method prototypes and data types that need to be serialised

The procedure to reverse engineer a SOAP web service is as follows

  1. Obtain the WSDL file describing the service
  2. Run SVCUTIL.exe on the WSDL to generate the interface class source and the output.config
  3. Inherit the interface class in your service class
  4. Implement the action methods in your service class
  5. Incorporate the channel binding in output.config into the web.config file

Let’s look at a simple example. A soap service that returns a GUID.

First create an empty WSDL service:

In visual studio 2013

File -> new project ->WCF->WCF Service Application

 

Copy the WSDL file from the original service to into the directory containing the service source files

Now run svcutil on the wsdl file.

Note that the ScvUtil.exe resides in the .NET tools directory. Two files will be created, guidService.cs and output.config

The guidService.cs file contains the service and operations contracts decorations and the interface class for the service.Replace the service and Operations contracts in your Iservice.cs file with these definitions.

This is where a lot of time and pain is saved.(If any of the method definitions has a misspelt parameter this will cause an error when interfacing to the service and can be quite difficult to find. Same goes for the Namespace and configuration name)

Incorporate the binding information from the output.config file to your web.config file.

Finally in your service.svc file inherit the interface class and implement the methods required

And now you have your own implementation of the original web service.

To test your reverse engineered web service take the original wsdl and load this into SOAP UI, if you can connect and operate the service with the original wsdl you are worthy J

Hope this helps with your testing J

 

 

 

Capturing Traffic from Genymotion Emulator

Capturing IP traffic from a Genymotion Android VM under windows is not trivial. The Genymotion virtual network adaptor communicates independently of the IP Stack of the host OS. Additionally network analyser packages running under windows such as Wireshark rely on WinPCAP library which attaches to the windows IP stack. (Also Loopback traffic doesn’t exercise the IP stack hence is not detected by packages such as Wireshark.)

The way to capture traffic is to configure an interception proxy on the Genymotion VM. This example will use fiddler as the proxy.

Configuring a Proxy on Genymotion

Select WIREDSSID from the Settings Menu

Click on WiredSSID and hold down for 2 seconds to get Modify Network Window

Select Modify Network then manually configure proxy. The IP address will be the IP address of the host machine

Port 8888 is the default proxy port for Fiddler and most other proxy software

Save this. Now all http traffic will be forced to the proxy where it can be intercepted.

Also by proxying to a physical address traffic is forced through the Hosts TCP/IP stack allowing packages such as Wireshark to capture traffic also.

Fiddler is a Http capture tool from Telerik and can be found at http://www.telerik.com/fiddler

To view the traffic in fiddler ensure that the “Allow remote computers to connect” option under Connections is selected

Now you will be able to see Requests from the Dalvik (Android VM) in fiddler. Enjoy debugging your awesome apps.

 

 

Interfacing physical OTG devices to Genymotion Android VM

Using a virtual Android device does NOT mean you can’t interface real devices to it.

In this example I will show how to configure a Genymotion Android VM to interface to an ANT+ usb dongle and receive Heart rate information from an ANT+ enabled heart rate monitor

The key to interfacing USB devices to a Genymotion Android VM is to be familiar with the USB device filters of the Oracle Virtual Box Product which is the host for the Genymotion VM

Open Virtual Box, Highlight the target VM and Select Settings

Select USB

Select USB and make sure USB Controller is enabled. You will see a list of detected devices in the Device Filter Dialog, Select the Device you wish the virtual machine to have access to.

The usb filter works by matching various parameters such as vendor ID, Product ID and Serial number.

This can be seen by editing the Filter as follows

The filter will list the parameters that need to be matched to connect the device. The trick is to use as few parameters as possible as the filter can be buggy!

 

Once the filter is enabled, fire up the Genymotion machine, THEN plug in the device

I use the “USB device info” App installed on the VM to check the status of the USB device

AS you can see the device is seen by the Genymotion VM and can be used with the appropriate App

Have fun using your emulator as USB enabled device.

ADB over Wi-Fi with Genymotion Android Emulator

I do my android development under a Ubuntu Virtual Machine for ease of use with GIT and Gradle. However I am unable to run the Genymotion Emulator within this virtual machine as the Oracle virtual box hypervisor is problematic under a VMware virtual Machine.

To overcome this problem I have configured the Genymotion Emulator to use the ADB (Android Debugger) over Wi-Fi. The emulator runs in the windows host and interfaces to the Ubuntu development VM using the virtual network.

Setting up the Genymotion Emulator

The steps are:

  1. Configure a second network interface as a bridge
  2. Install Google Play
  3. Install an ADB over WIFI application

After installing the Genymotion/VirtualBox bundle under windows execute Virtual Box

adb1

 

Select Settings and configure Adaptor 2 as shown. The adaptor will be configured as a Bridged Adaptor. Select a physical interface that has connectivity to the development machine. In my case I am using the WIFI interface as it provides connectivity between the emulator and the VMware Host OS (since it is the DHCP client of my router)

adb2

Enable debug via usb by switching on developer options.  Developer options are enabled from Settings->about tablet by tapping the build number 7 times.

From developer options select debug via usb

adb2-5

 

We need to install Google Play so that a suitable USB over WI-FI application can be sourced.

Download the gapps rom for the version of Android. I use AndroidFileHost.com https://www.androidfilehost.com/?fid=23311191640114013

This can be installed by simply dragging the file onto a running Genymotion emulator

Follow the prompts and reboot

adb3

adb4

adb5

 

Now use Google Play to download an ADB to WIFI application. I have found the best results using ADB Wireless Pro as it is easy to quickly enable and disable and allows switching back to usb based adb if required. Install ADB Wireless Pro and grant Superuser access.

adb7

adb8

Check that adb over WI-FI is enabled.  Make note of the ip address reported. In this case it is 192.168.1.160

replace this with your IP address when running the adb commands.

adb9

The Emulator is now ready to connect with the developer machine.

For the development machine enter the command “adb connect 192.168.1.100”. Adb listens on port 5555 by default.

If required the port can be changed on the emulator via the ADB wireless pro menu.

Example of a successful connection below

adb10

To disconnect the debugger from the emulator use the “adb disconnect 192.168.1.160”

adb11

 

Finally Desktop snapshot with an Emulator being controlled from a Ubuntu Development VM. Enjoy you sick puppies

adb13

 

 

FireBootCamp Retrospective

I have just completed 9 weeks of Firebootcamp, the code camp from hell which makes the SAS selection course appear like a ride on a Vespa with a decafe soya  late on the handle bars!
Despite this self inflicted torture, I have come to realise that no process, not even Agile is as effective as the synergies found in a well functioning team. I am reminded of the words of an old boss, the former CTO of Telstra Bigpond who told me “if you get the people stuff right, the technology will take care of itself”.

My team mates, Ludvik, Andrew and Michael all collaborated and shared. When one was down the others pulled him up, we shared each others pain and triumph.

Agile,Waterfall,whatever… , what is really important is working as a team for in working as a team the true nature of humanity is amplified.

logo

Is your website healthy?

You all know the feeling, you are stirred from sleep by the support phone. Arrrgggghhhhhh says the Director! The website is not working fix it.

Enterprise websites are living things that exist in harmony with external entities such as databases, pricing feeds and identity services and like a person often the problems are external rather than internal,

HealthCheck is a package released by SSW, to provide a framework to list all checks needed to determine the health of websites components. It uses SignalR which extends multicast functionality into the web space and handles the degradation from Web Sockets for earlier browsers using IP keep alive packets.

In a sense it extends the concept of unit tests to interfaces and provides a web interface for such tests.

Examples of Healthcheck pages  below:

staging database check

local test

What Web Programming students can learn from Air France 447

Air France flight 447 was lost over the Atlantic Ocean on 1st June 2009. The autopilot disengaged from the flight management system due to icing of the Pitot tubes, these tubes measure airspeed using pressure differentials.. Without this measurement the autopilot was not able to correctly control the aircraft and defaulted back to manual control. The plane later crashed in part due to the aircrew lacking experience in flying without an autopilot. In fact if the pilots just left the controls alone the aircraft computer would have come back online in a few minutes once the pitot tubes cleared, however many hours of only flying an aircraft on autopilot caused the pilots to react incorrectly to a simple problem

As a Web Developer I have logged most of my recent programming hours using Visual Studio with the MVC template. The rich Visual Studio environment automatically provides me with integrated Bootstrap, Jquery, and HTML view engine RAZOR. Like an autopilot on an Airbus Visual Studio provided me with a stable platform without needing to be aware of libraries and style concerns.  Like the Airbus pilots I was found lacking in solving simple problems with scripting and styling as I had relied entirely on the framework to manage my applications previously.

Airline pilots often will fly small Cessna class aircraft to maintain the basic skills of flying manually, without an autopilot and with an awareness of the environment around them.’

Similarly I have found as a Web Developer that using simple “Sandboxes” such as “Jsfiddle” provide a means to hone the fundamental skills of developing a web page.

jsfiddle

 

The Jsfiddle enviroment provides seperate panes to focus attention on HTML,CSS and scripting issues and provides an awareness of what libaries have been added on the left pane. Most importantly as a web based tool, many programmer communities such as “stack overflow” use Jsfiddle  examples to post example problems and solutions . An example of one of these  “fiddles” can be found on my jsfiddle account at http://jsfiddle.net/bigcraig/82Gg7/ . This is an example of using the built in “event styles” to provide interactive highlighting on Kendo grids, I recommend JsFiddle to anyone wishing to greatly accelerate their web development learning.

 

 

 

 

 

Configure a custom domain with Windows Azure

 Migrating a free azure website to a custom domain IS EASY

These are the simple steps

  1. Migrate the free website to a shared website
  2. Use the configure option to manage domains
  3. Enter your custom domain
  4. Enter the cname and A records on your DNS
  5. Save your domain into Azure
  6. Browse your new website

Lets take my Blog as a detailed example

Log into windowsazure.com management portal and select the free website

azure1

 

Select scale and move your website to shared, select only a single instance to minimise your running costs.

azure2

 

azure3

Now Select configure then scroll to manage domains

azure4

azure6

You will be presented with the following dialog, pay attention to the IP address supplied and the instructions on CNAME records circled

azure7

Now I wish to explain some DNS theory. DNS (Domain Name Service), has only one purpose in life, that is to convert a hostname, e.g. yourname.com, into an IP address. The first way to do this is to have a DNS record that states the IP address for a particular hostname. This is called an “A” record.

So when DNS gets a request for the IP address of “yourname.com” it returns the IP address specified in the “A” record.

The second type of record is a CNAME (Canonical Name) record. This record redirects your hostname to a second domain name and the IP address of this second domain name is provided for use. In my example I redirect my domain scrumsofanarchy.com to my azure website domain scrumsofanarchy.freeazurewebsites.net using a CNAME record.

Before we can point our custom domain name to the Azure website Microsoft requires verification that we in fact control the domain by requiring certain CNAME records be configured. In my example I need the following CNAME records

azure8

Note also I have added an A record using the IP address specified in the manage domains dialog

To make these changes to your domain you will need to log into the domain registar for your domain and select DNS settings. Just select add new records and add the CNAME and A records required.

Now once this is done we can go back to the azure console and save the domain. If the domain configuration is correct we should get the following

azure10

Notice the nice green tick. If you get an error check the spelling of your cname records. Not that I have added both scrumsofanarchy.com and www.scrumsofanarchy.com to my custom domains.

THAT’S IT!!!!!! Now you have a professional looking domain for you Blog and no one knows where It is being hosted. Also note the re-directions are invisible to the user 🙂

 

One last point. By using CNAME redirects for scumsofanarchy.com to scrumsofanarchy.azurewebsites.net . I have bulletproofed my website from any change to IP address as the CNAME lookup will provide the correct IP Address. Some other forums have noted that the IP Address of the Azure Website MAY change if the Azure Virtual machine is restarted or moved.

If you require any further assistance please leave a comment

Craig 🙂