Cluster Computing in Android Based Devices

Dept. of Computer Science and Engineering,

Indian Institute of Technology Roorkee

Cluster Computing in Android based Devices in Collaborative Environment

B.Tech Project Mid-Term Evaluation, April 2014 Group Members

Kari Dharani Kumar Vishruth Reddy Sumit Badwal Sumit Kumar

10114024 10114030 10114040 10114041

Supervisor

DR. P. SATEESH KUMAR Assistant Professor,

Dept. of Computer Science and Engineering, IIT Roorkee.

1

Table of Contents Problem Statement ....................................................................................................................................... 3 Introduction .................................................................................................................................................. 3 Work Done .................................................................................................................................................... 4 Cluster setup on x86 ............................................................................................................................. 4 Functioning of Android Application ...................................................................................................... 4 Modules ........................................................................................................................................................ 6 User Identification ................................................................................................................................ 6 Splitting Text File ................................................................................................................................... 6 Searching ............................................................................................................................................... 7 Snapshots of Application ...................................................................................................................... 8 Performance Evaluation.............................................................................................................................. 11 Measuring Speedup ............................................................................................................................ 11 Network Performance (using NETPerf) ............................................................................................... 11 Future Work ................................................................................................................................................ 12 Conclusion ................................................................................................................................................... 13 References .................................................................................................................................................. 14

2

Our topic “Cluster Computing in Android based devices” focuses on two points load balancing and high Availability. The program or task given to an embedded system like mobile or digital TV can be parallelized for better performance and throughput. The program needs to be distributed into separate tasks and then given to each node in the cluster. This completes the task in less time and proves effective. Also working in a cluster, the overall system becomes computationally powerful. Moving on to our project, we chose to do the same work which is done on the x86 machines to ARM processors. To be precise, we want to get the same work done on the ARM based embedded systems like Mobile phones, Digital TVs which runs Android, or any flavour of Linux.

: In cluster computing, a collection of stand-alone computers are interconnected to form a single integrated computing resource, which is able to improve performance and availability than using a single computer. Most Common uses are load balancing  and providing high availability. Load Balancing - a single workload (e.g. a computation) is shared by several computers that are linked together, which work as a single unit. Any of the workloads coming to the system are distributed among the computers in the cluster, such that the work is balanced among them. This improves the performance of the whole system. High Availability  - redundant nodes are provided to make sure that the service provided by the cluster is always available (even when some system components fail). Clusters can achieve a great improvement in performance when compared to the price. •

•

Our project focuses on Mobile Phone Cluster which had to be implemented on Android platform. The phones will run the Android Application and will discover each other, transfer data, send messages to each other over the same Wi-Fi network. A task is generated or received by the master node which is already in the same Wi-Fi network. The task will be transferred and assigned through the same method to other mobile slaves. Each slave will then send result back to the master. Hence our system model has more features over a computer cluster. Mobility - the mobile devices can be relocated anywhere as they are not dependent upon the location. They just need a network to connect each other which in our case is Wi-Fi. Scalability - the system is highly scalable since the mobile devices can be added or removed when needed. When load on the system would be high then new Android devices running the same application will be introduced in the Wi-Fi network. Decentralization - the slaves are treated similar and power to process is given to each slave by the master. •

•

•

3

Fig: 1 Setting up the cluster

The devices running the application can detect other devices running the same application over the same Wi-Fi network. This is achieved by using UDP-broadcast discovery and then use a TCP/IP based protocol stack to create a reliable, local, peer-to-peer communications network. UDP stands for User Datagram Protocol and the broadcast packets are sent to entire subnet. (e.g.: 192.168.7.001 to 192.168.7.254). Each node transmits UDP broadcast periodically and parses broadcast messages from other nodes to discover all the nodes on the same subnet. TCP stands for Transmission Control Protocol and unlike UDP it is used for creating reliable connections. In a TCP connection acknowledgement is sent upon data reception. It contains 3 phases Connection Establishment Send Data Connection Termination • • •

4

The application can send data like text messages and images with selected members of the Wi-Fi network. There are two ways to send data through channels: Public Channels - The mobile device or node is automatically added to the public channel as soon as it start’s discovery. This channel doesn’t restrict any node. Private Channels - The nodes can create their own channels by naming them. So if any node joins a channel of some name and other nodes also join the channel with the same name then this channel becomes private.

6

1

Public Channel

Channel A

5

2

Channel B

4

3

5

Fig 2. Mobile Channels

Our project can be divided into different modules dealing with the proper communication and distribution of tasks. Our task is specifically designed to search for key-value pairs in a text file. This task is very computationally intensive when there are very large number of entries. In our case we will split the text file into smaller files and distribute these text files to each node which searches and then give back the result. A mobile device will be having unique identification code like IP or MAC Address of the Wi-Fi Adapter. This ID of device can be used for the communication but it will be difficult for a user to identify a particular device. The application will involve registration based entry. Each device will have a name and that name will be sent along with the ID. So the user will be able to which mobile device it is sharing data. Basically it’s a hash map of ID and the name. DEVICE ID

VALUE

Mt7kR1R!l*mdC94E

Sumit-xperia

hKRjtKmj#kh1FE47

vishruth-galaxy

sKTh3Kde$gf3N89O

Badwal-note3

cPmchsbj&bc5FgC9

Dharani-duos

ID and keys are assigned m-bit identifier using consistent hashing because both keys and IDs (IP addresses) are uniformly distributed and in the same identifier space. Consistent hashing is also necessary to let nodes join and leave the network without disruption.

There is a mapping of the key onto a node. Both keys and nodes are assigned an m-bit identifier. For nodes, this identifier is a hash of the node's IP address. For keys, this identifier is a hash of a keyword, such as a file name. It is not uncommon to use the words "nodes" and "keys" to refer to these identifiers, rather than actual nodes or keys.

Splitting a text file into multiple smaller text files in order to get the work done by as many devices as possible. The Advantage is the use of multiple devices instead of single entity to finish the task for us. The splitting can be done based on the size of the file or no of lines to be present in the file. This Algorithm works in a similar way to Split command in Linux. Files with similar names are created dynamically depending on the size of the file. File Descriptors plays a main role in the access permissions to the file. Before reading or writing into the file it must be opened. Never forget to close the file descriptor of the file after completion of the work. A text file with large size is defragmented into small chunks of data as small individual text files. We can choose the length of the line and the number of lines in the file which helps in the Performance of the Algorithm. Run the Loop until no more line exist. The line size can be customized and a buffer is used to write the data into the file. When the number of lines in a text

6

file has been reached which is known with the help of line counter , the file is closed and a new text file is created and the same process is repeated. Pseudocode: Cr eat e 2 f i l e Poi nt er s 1 to r ead and ot her t o Wr i t e Set a Char Buf f er t o Max Li ne Si ze Cr eat e an Ar r ay t o St or e new Spl i t Fi l es Set Fi l eCount er and Li neCount er t o 1 Open t he Fi l e t o Read Cr eat e and open a New Spl i t Fi l e Loop t hr ough al l l i nes of t he f i l e I f Li neCount er equal s Max no of l i nes i n a Spl i t f i l e, Cl os e t he Spl i t Fi l e Reset t he Li neCount er t o 1 I ncr ement t he Fi l eCount er Cr eat e and open a New Spl i t Fi l e Br eak to t he next i t er at i on of t he l oop El se Wr i t e t he l i nes t o t he Out put f i l e End i f End Loop Cl ose t he Read Fi l e

Cr eat e an ar r ay t o st or e t he l i ne number s Set Resul t =1 t o dynami cal l y i ncr ease t he l engt h of ar r ay Open t he f i l e t o Read sel ect on t he new l i ne Loop t hr ough each l i ne of f i l e, sear chi ng f or a mat chi ng st r i ng I f a mat ch i s f ound, St or e t he l i ne number i n an ar r ay I ncr ement t he Resul t by 1 Br eak t o t he next i t er at i on of t he l oop El se ' i f no el ement i n t he ar r ay has a mat ch f or t he cur r ent l i ne Set t he f i r st bl ank el ement = t he cur r ent l i ne End i f case def aul t : ' i f t he l i ne cont ai ns anyt hi ng ot her t han a dat e Append t he l i ne t o el ement Marker ' i . e. , t he most r ecent l y used el ement i n t he arr ay End sel ect Wend Set t he f unct i on = sear ch_i n_f i l e End t he f unct i on

7

Fig 3. First Screen

The first screen checks for Wi-Fi network. If there is no Wi-Fi network then it shows error and user can’t proceed to next screen of Discovery Process.

Fig 4.Chord Discovery

In the next screen the user can start the discovery of his Android device and get access to channels. The “Start Discovery” button starts and stops the discovery. After Start Discovery has been tapped, the android device sends UDP broadcast packets continuously to find other devices on the same Wi-Fi network. Also the application parses the broadcast messages received from other devices to get the list of devices already present in the network.

8

Fig 5 Public Channel joined

The android device has received UDP packets from another device. Hence it has joined the public channel. Here the IDs of the other phones are displayed.

Fig 6. Private Channel joined

The private channel is a form of secure channel for sharing data. The channel is joined by giving a name to the channel. So, to make it secure user can give a unique name. Private channel has features similar to public channel with the difference of just the name based entry.

9

Fig 7. All Available nodes

After pressing the “more” button user gets this screen where all the mobile devices of the channel are present. Both the “ public channel ” and “ joined channel ” provide the same screen and features. User will be able to see the channel name at the top to which he/she wants to share data. The nodes can be selected using the checkboxes. To attach a file to share with nodes there is a “ File” button. The file types has been restricted to text and images only.

Fig 8. Data Transfer

For now, the data sharing is displayed in the form of chat to keep record.

Eclipse IDE Android SDK ZeroMQ (ØMQ) - It is a high performance asynchronous messaging library allowing us to design a complex communication system easily. We could have used Berkeley socket interface but maintaining raw sockets is difficult and cumbersome when we have to build a scalable system. Really fast and simple (8M msgs/sec, 30µsec latency) [2]. • • •

10

We can find the performance gain (speedup) obtained by parallelizing an application over N processors.

‘Speedup’: Ratio of time it takes to solve a problem on a single processor T(1) to the time it takes to solve the problem on N processors T(N). For single processor speedup is   =

 (1)  ()

Eq. [1]

Upon parallelization, the run time is dependent on  - serial run time  - parallel run time Now, for N processors, speedup is  =

 + 

Eq. [2]

  + 

NETPerf is a benchmark for measuring network standards regarding various aspects of the network. For our work we restrict ourselves to monitor the network load on the cluster during communication and file transfers. For communication standards, we measure the packet drop rate, latency. For file transfer, we try to maximize the bandwidth by finding the appropriate packet size.

11

We have so far achieved secure connection between multiple android device hosts and used it to send messages and transfer files securely to one or multiple devices. The connection among the devices is established using WiFi and uses ZeroMQ(ØMQ) library to communicate with them. Now we aim at using this underlying connection mode to implement cluster of android based devices which will be running a task that requires high computational resources in parallel on all the connected nodes. We need to identify each mobile device and provide a security feature like a registration or some password to ensure that authentic users can access the application. Also, the registration would provide identity to the devices so that users can know which device is trying to connect and share data. Since we are able to connect devices and communicate among them, our main focus will be to distribute the heavy task among devices without compromising power and connection cost. We will be doing the performance evaluation of Beowulf Cluster. Thereafter comparing the cost efficiency, performance efficiency. Multiple Android Nodes Available

Connect over Wi-Fi using ZeroMQ

All Nodes connected in Cluster

Task request sent to Master

High Computational Task distributed among all nodes Result from each node

Results of Every Node sent to Master to get final Result Final Result sent Fig 9. Future Course of Work.

Requesting Node received Final Result 12

The first phase involved a deep research in cluster computing and its advantages and disadvantages. We had to find out the usage of cluster and the kind of tasks it can perform efficiently. In the next phase we did some implementation of Beowulf Cluster on our laptops and saw how significantly it could use resources to complete a set of tasks. Also we implemented our research on Android Application. The Android Application uses Wi-Fi network to communicate with the other nodes. In the next phase we will improve its functioning by adding features of automatic task assignment and then result compilation.

13

1. http://www.mpich.org/ 2. http://www.slideshare.net/pieterh/overview-of-zeromq 3. An evaluation of the system performance of a beowulf cluster by Karl Johan Andersson, Daniel Aronsson and Patrick Karlsson - http://www.nsc.liu.se/grendel/ . 4. Byobu - Building a simple Beowulf cluster with Ubuntu: http://byobu.info/article/Building_a_simple_Beowulf_cluster_with_Ubuntu/ . 5. ZeroMQ: http://zguide.zeromq.org/page:all/ . 6. Chord protocol- http://en.wikipedia.org/wiki/Chord_%28peer-to-peer%29/

14