SAMPLE LAB MANUAL
GRID AND CLOUD LAB Prepared By Mr. R.Ganesh
Private Circuati!n Ganeshravi"#$%&'(ai.c!(
INDEX EXPERIMENT No
Date
Name of the Experiments RID !"MP#TIN $%&
$
Deve!p a ne) *e+ Service ,!r Cacuat!r.
-
Deve!p ne) OGSAc!(piant *e+ Service.
/
Usin' Apache A0is deve!p a Grid Service.
"
Deve!p appicati!ns usin' 1ava !r C2C33 Grid APIs
4
Deve!p Deve!p secured secured appica appicati!n ti!nss usin' usin' +asic +asic securit security y (echanis(s avaia+e in G!+us 5!!6it. Deve!p a Grid p!rta8 )here user can su+(it a 9!+ and 'et the resut. I(pe(ent it )ith and )ith!ut GRAM c!ncept
7
:
;ind pr!cedure t! run the virtua (achine !, di,,erent c!n,i'urati!n. Chec6 h!) (any virtua (achines can +e utii
%
;ind pr!cedure t! attach virtua +!c6 t! the virtua (achine and chec6 )hether it h!ds the data even a,ter the reease !, the virtua (achine. Insta a C c!(pier in the virtua (achine and e0ecute a sa(pe pr!'ra(.
$#
Sh!) the virtua (achine (i'rati!n +ased !n the certain c!nditi!n ,r!( !ne n!de t! the !ther
$$
;ind ;ind pr!c pr!ced edur uree t! inst insta a st!r st!ra' a'ee c!nt c!ntr! r! er er and and interact )ith it.
$-
;ind pr!cedure t! set up the !ne n!de >ad!!p custer
$/
M!unt the !ne n!de >ad!!p custer usin' ;USE
$"
*rite a pr!'ra( t! use the API?s !, >ad!!p t! interact )ith it
$4
*rite *rite a )!rdc!unt pr!'ra( t! de(!nstrate the use !, Map and Reduce tas6s
=
Pg. No.
Faculty Signature
Ex No' ( De)elop a ne* +e, Ser)ice for !alculator D%TE'
"&-E!TIE'
5! deve!p a ne) *e+ service ,!r Cacuat!r appicati!ns.
PR"!ED#RE'
*hen y!u start G!+us t!!6it c!ntainer8 there )i +e nu(+er !, services starts up. 5he service ,!r this tas6 )i +e a si(pe Math service that can per,!r( +asic arith(etic ,!r a cient. 5he Math service )i access a res!urce )ith t)! pr!perties@ $. An inte'e inte'err vaue vaue that can can +e !perat !perated ed up!n +y +y the servic servicee -. A strin' vaues vaues that h!ds h!ds strin' strin' descri+in' descri+in' the ast ast !perati!n !perati!n 5he service itse, )i have three re(!tey accessi+e !perati!ns that !perate up!n value@
a add8 add8 that that adds adds a t! the res!urce pr!perty value. + su+tract su+tract that that su+tracts su+tracts a ,r!( the res!urce pr!perty value. c 'etaueRP 'etaueRP that returns the current current vaue !, value. Usuay8 the +est )ay ,!r any pr!'ra((in' tas6 is t! +e'in )ith an !vera descripti!n !, )hat y!u )ant the c!de t! d!8 )hich in this case is the service inter,ace. 5he service inter,ace descri+es h!) )hat the service pr!vides in ter(s !, na(es !, !perati!ns8 their ar'u(ents and return vaues. A 1ava inter,ace ,!r !ur service is@ pu,lic interface Math / pu,lic )oi0 a001int a23 pu,lic )oi0 su,tract1int a23 pu,lic int getalueRP123 4
It is p!ssi+e t! start )ith this inter,ace inter,ace and create the necessary *SDL ,ie ,ie usin' the standard *e+ service t!! caed 1ava-*SDL. >!)ever8 the *SDL ,ie ,!r G5 " has t!
incude detais !, res!urce pr!perties that are n!t 'iven e0picity in the inter,ace a+!ve. >ence8 )e )i pr!vide the *SDL ,ie.
Step 1 Getting the Files Files
A the reuired ,ies are pr!vided and c!(es directy ,r!( $F. 5he MathService s!urce c!de ,ies can +e ,!und ,r!( http@22))).'t"+!!6.c!( http@22))).'t"+!!6.c!(2d!)n!ads2't"+!!6e0a(pes.tar.'< http@22))).'t"+!!6.c!(2d!)n!ads2't"+!!6e0a(pes.tar.'< A *ind!)s
+SD$ ser)ice interface 0escription file -- The WSDL service interface description file is provided within the GT4services GT4services folder at: GT4Services\schema\eamp GT4Services\schema\eamples\!athService les\!athService"instance\!ath# "instance\!ath#wsdl wsdl
5his ,ie8 and discussi!n !, its c!ntents8 can +e ,!und in Appendi0 A. Later !n )e )i need t! (!di,y this ,ie8 +ut ,irst )e )i use the e0istin' c!ntents that descri+e the Math service a+!ve. Service c!de in 1ava ;!r this assi'n(ent8 +!th the c!de ,!r service !perati!ns and ,!r the res!urce pr!perties are put in the sa(e cass ,!r c!nvenience. M!re c!(pe0 services and res!urces )!ud +e de,ined in separate casses. 5he 1ava c!de ,!r the service and its res!urce pr!perties is !cated )ithin the G5"services ,!der at@ T5ser)ices6org6glo,us6exam T5ser)ices6org6glo,us6examples6ser)ices6c ples6ser)ices6core6first6impl6MathSer ore6first6impl6MathSer)ice.7a)a )ice.7a)a .
Deployment Descriptor Descriptor -- 5he dep!y(ent descript!r 'ives severa di,,erent i(p!rtant sets !,
in,!r(ati!n in,!r(ati!n a+!ut the service !nce it is dep!yed. It is !cated )ithin the T5ser)ices ,!der at@ T5ser)ices6org6glo,us6examples6ser)ices6core6first60eploy8ser)er.*s00 .
Step 9 : &uil0ing the Math Ser)ice
It is n!) necessary t! pac6a'e a the reuired ,ies int! a GAR Grid Archive ,ie. 5he +uid t!! ant ,r!( the Apache S!,t)are ;!undati!n is used t! achieve this as sh!)n !verea,@ Generatin' a GAR ,ie )ith Ant ,r!( http@22'dp.'!+us.!r'2't"tut!ria2(utipeht(2ch#/s#".ht( Ant is si(iar in c!ncept t! the Uni0 (a6e t!! +ut a 9ava t!! and HML +ased. Buid scripts are pr!vided +y G!+us " t! use the ant +uid ,ie. 5he )ind!)s versi!n !, the +uid script ,!r MathService is the Pyth!n ,ie caed glo,us8,uil08ser)ice.py8 )hich hed in the T5ser)i T5ser)ices ces direct!ry. 5he +uid script ta6es !ne ar'u(ent8 the na(e !, y!ur service that y!u )ant t! dep!y. dep!y. 5! 5! 6eep )ith the na(in' c!nventi!n in $F8 this service )i +e caed first. In the Client Window8 run the +uid script ,r!( the T5ser)ices direct!ry )ith@ glo,us8,uil08ser)ice.py glo,us8,uil08ser)ice.py first
5he !utput sh!ud !!6 si(iar t! the ,!!)in'@ &uil0file' ,uil0.xml . . . . . &#I$D S#!!ESSF#$ Total time' ; secon0s
Durin' Durin' the +uid +uid pr!cess8 pr!cess8 a ne) direct!ry direct!ry is created created in y!ur T5Ser)ic T5Ser)ices es direct!ry that is na(ed ,uil0. A !, y!ur stu+s and cass ,ies that )ere 'enerated )i +e in that direct!ry and its its su+d su+dir irec ect! t!ri ries es.. M! M!re re i(p! i(p!rt rtan ant ty y8 ther theree is a GAR GAR Gri Grid d Arch Archiv ive e ,ie ,ie ca caed ed org
every ,ie that is needed t! success,uy success,uy dep!y y!ur Math Service int! the G!+us c!ntainer c!ntainer. 5he ,ies c!ntained in the GAR ,ie are the 1ava cass ,ies8 *SDL8 c!(pied stu+s8 and the dep!y(ent descript!r.
Step = : Deploying the Math Ser)ice
I, the c!ntainer is sti runnin' in the C!ntainer *ind!)8 then st!p it usin' C!ntr!C. 5! dep!y the Math Service8 y!u )i use a t!! pr!vided +y the G!+us 5!!6it caed glo,us8 0eploy8gar. In the Container Window 8 issue the c!((and@ glo,us80eploy8gar glo,us80eploy8gar org
Success,u !utput !, the c!((and is @
5he service has n!) +een dep!yed. Chec6 service is dep!yed +y startin' c!ntainer ,r!( the Container Window @ !u sh!ud see the service caed MathSer)ice.
Step 5 : !ompiling the !lient
A cient has aready +een pr!vided t! test the Math Service and is !cated in the T5Ser)ices direct!ry at@ T5Ser)ices6org6glo,us6example T5Ser)ices6org6glo,us6examples6clients6MathSer)ice s6clients6MathSer)ice
and c!ntains
!u sh!ud see the service caed MathSer)ice.
Step 5 : !ompiling the !lient
A cient has aready +een pr!vided t! test the Math Service and is !cated in the T5Ser)ices direct!ry at@ T5Ser)ices6org6glo,us6example T5Ser)ices6org6glo,us6examples6clients6MathSer)ice s6clients6MathSer)ice
and c!ntains the ,!!)in' c!de@ pac>age org.glo,us.examples.clients.Ma org.glo,us.examples.clients.MathSer)ice
Trace123 4 4 4 *hen the cient is run ,r!( the c!((and ine8 y!u pass it !ne ar'u(ent. 5he
ar'u(ent is the URL that speci,ies )here the service resides. 5he cient )i create the end p!int rer,erence and inc!rp!rate this th is URL as the address. 5he end p!int re,erence is then used )ith the getMathPo !+9ect t! getMathPortT rtTypeP ypePort ort (eth!d !, a MathSer)i MathSer)ice%0 ce%0res ressing$ sing$ocato ocatorr !+9ect !+tain a re,erence t! the Math inter,ace p!rt5ype. 5hen8 )e can appy the (eth!ds avaia+e in the service as th!u'h they )ere !ca (eth!ds N!tice that the ca t! the service add and su+tract (eth!d cas (ust +e in a Jtry /4 catch12/4K +!c6 +ecause a JRe(!teE0cepti!nK
(ay +e thr!)n. 5he c!de ,!r the JMathSer)ice%00ressing$ocator K is created durin' the +uid pr!cess. 5hus y!u y! u d!nt have t! )rite it
$a% Settting the &lasspath
5! c!(pie the ne) cient8 y!u )i need the 1AR ,ies ,r!( the G!+us t!!6it in y!ur CLASSPA CLASSPA5>. D! this +y e0ecutin' the ,!!)in' c!((and in the Cient *ind!)@ G$"&#S<$"!%TI"NG6etc6glo,us80e)el8en).,at
!u can veri,y v eri,y that this sets y!ur CLASSPA CLASSPA5>8 +y e0ecutin' e0ecutin ' the c!((and@ echo G!$%SSP%THG G!$%SSP%THG
!u sh!ud see a !n' ist !, 1AR ,ies. Runnin' 6gt56etc6glo,us80e)el8en).,at !ny needs t! +e d!ne once ,!r each Client Window Window that y!u !pen. It d!es not need t! +e d!ne each ti(e y!u c!(pie. $'% &ompiling &lient
Once y!ur CLASSPA5> has +een set8 then y!u can c!(pie the Cient c!de +y typin' in the ,!!)in' c!((and@ 7a)ac :classpath :classpath ,uil06classes6org6glo,us6example ,uil06classes6org6glo,us6examples6ser)ices6cor s6ser)ices6core6first6impl6' e6first6impl6' G!$%SSP%THG org6glo,us6examples6clients6MathSer org6glo,us6examples6clients6MathSer)ice
Step : Start the !ontainer for your Ser)ice
Restart the G!+us c!ntainer ,r!( the Container Window )ith@ glo,us8start8containerr 8nosec glo,us8start8containe
i, the c!ntainer is n!t runnin'.
Step : Run the !lient
5! start the cient ,r!( y!ur T5Ser)ices direct!ry direct!ry88 d! the ,!!)in' in the Client Window8 )hich passes the GS> !, the service as an ar'u(ent@ 7a)a 8classpath ,uil06classes6org6glo,us6examples6ser)ices6core6first6impl6'G!$%SSP%THG org.glo,us.examples.clients.MathS org.glo,us.examples.clients.MathSer)ice
)hich sh!ud 'ive the !utput@ Current vaue@ $4 Current vaue@ $#
Step J : #n0eploy the Math Ser)ice an0 Kill a !ontainer
Be,!re )e can add ,uncti!naity t! the Math Service Secti!n 48 )e (ust undep!y the service. In the Container Window 8 6i the c!ntainer )ith a C!ntr!C. 5hen t! undep!y the service8 type in the ,!!)in' c!((and@
glo,us8un0eploy8gar org
)hich sh!ud resut )ith the ,!!)in' !utput@ #n0eploying gar... Deleting C. . . #n0eploy successful 6 Adding Functionality to the Math Service
In this ,ina tas68 y!u are as6ed t! (!di,y the Math service and ass!ciated ,ies s! the srvice supp!rts the (utipicati!n !perati!n. 5! d! this tas68 y!u )i need t! (!di,y@ Service c!de MathSer)ice.7a)a *SDL ,ie Math.*s0l2 5he e0act chan'es that are necessary are n!t 'iven. !u !u are t! )!r6 the( !ut y!urse,. !u )i need t! ,uy understand the c!ntents !, service c!de and *SDL ,ies and then (!di,y the( acc!rdin'y. Appendi0 A 'ives an e0panati!n !, the i(p!rtant parts !, these ,ies. eep a ,ie na(es the sa(e and si(py redep!y the service a,ter)ards. a,ter)ards. !u !u )i as! need t! add a c!de t! the cient c!de !lient.7a)a t! test the (!di,ied service t! incude (utipicati!n.
RES#$T'
Ex No' 9 De)elop ne* "S%8compliant +e, Ser)ice D%TE'
"&-E!TIE'
5! deve!p a ne) OGSAc!(piant )e+ service.
PR"!ED#RE'
*ritin' and dep!yin' a *SR; *e+ Service is easier than y!u (i'ht thin6. !u 9ust have t! ,!!) ,ive si(pe steps $. De,ine De,ine the the service? service?ss inter,ac inter,ace. e. 5his 5his is d!ne d!ne )ith )ith WSDL -. I(pe( I(pe(ent ent the the servic service. e. 5his 5his is is d!ne d!ne )ith )ith Java. /. De,ine De,ine the the dep!y( dep!y(ent ent para(e para(eters ters.. 5his 5his is d!ne d!ne )ith )ith WSDD and JD! ". C!(pie C!(pie everyt everythin' hin' and and 'enerate 'enerate a GAR ,ie. ,ie. 5his 5his is d!ne d!ne )ith )ith Ant 4. Dep!y Dep!y servi service. ce. 5his 5his is as! d!ne )ith )ith a "#$ tool 5! run this pr!'ra(8 as a (ini(u( y!u )i +e reuired t! have instaed the ,!!)in' prereuisite s!,t)are a. D!)n!ad D!)n!ad the atest A0is- runti(e ,r!( the a+!ve in6 and e0tract it. N!) )e p!int Ecipse *5P t! d!)n!aded A0is- Runti(e. Open +in0o* 8L Preferences 8L +e, Ser)ices 8L %xis9 Emitter
Seect the A0is- Runti(e ta+ and p!int t! the c!rrect A0is- runti(e !cati!n. Aternativey at the A0is- Pre,erence ta+8 y!u can set the de,aut settin' that )i c!(e up !n the *e+ Services Creati!n )i
Cic6 ne0t d. See eect th the na( na(e %xis9+STest as the Dyna(ic *e+ pr!9ect na(e y!u can speci,y any any na(e na(e y!u pre,e pre,er8 r8 and and seec seectt the the c!n,i c!n,i'ur 'ured ed 5!( !(ca catt runti runti(e (e as the the tar' tar'et et runti(e.
Cic6 ne0t. e. Seect Seect the A0isA0is- *e+ *e+ servic servicee ,acet ,acet
Cic6 ;inish. ,. 5his 5his )i )i create create a dyna(i dyna(icc *e+ *e+ pr!9ect pr!9ect in in the )!r6+e )!r6+ench nch
'. I(p!rt the )tp2C!nverter.9ava )tp2C!nverter.9ava cass cass int! A0is-*S5est2src +e sure t! preserve the pac6a'e.
Buid the Pr!9ect8 i, its n!t aut! +uid. h. Seect C!nverter C!nverter.9ava8 .9ava8 !pen ;ie ;ie Ne) Other... Other... *e+ *e+ Services *e+ *e+ Service
Cic6 ne0t. i.
5he *e+ *e+ service service )i
M!ve the service scae t! Start ser)ice .
9.
Cic6 !n the +e, Ser)ice runtime in6 t! seect the A0is- runti(e.
Cic6 O. 6. Ensure that that the c!rrect c!rrect server server and service service pr!9ect pr!9ect are seected seected as dispayed dispayed +e!) +e!)..
Cic6 ne0t.
.
5his 5his pa'e is the the service. service.0( 0( seect seecti!n i!n pa'e. pa'e. i, y!u have have a cust!( cust!( service services.0( s.0(8 8 y!u can incude that +y cic6in' the &ro*se +utt!n. ;!r the (!(ent8 9ust eave it at the de,aut.
Cic6 ne0t. (. 5his pa'e is the the Start Server Server pa'e. It )i +e dispayed dispayed i, the server server has n!t +een started. Cic6 !n the Start Ser)er +utt!n. 5his )i start the server runti(e.
Cic6 ne0t. n. 5his pa'e pa'e is the the *e+ *e+ services services pu+icati!n pu+icati!n pa'e8 pa'e8 accept accept the de,auts. de,auts.
Cic6 ;inish. !. N!) N!)8 see seect ct the the %xis9+STest dyna(ic *e+ pr!9ect8 ri'htcic6 and seect Run Run As Run !n Server t! +rin' up the A0is- servet.
Cic6 Ne0t.
p. Ma6e sure y!u have the %xis9+STest dyna(ic *e+ pr!9ect !n the ri'hthand side under the C!n,i'ured pr!9ect.
Cic6 ;inish.
. 5his )i )i dep!y dep!y the A0is- server server )e+app )e+app !n the c!n,i'ured c!n,i'ured servet servet c!ntainer c!ntainer and and )i dispay the A0is- h!(e pa'e. N!te that the servet c!ntainer )i start up acc!rdin' t! the Server c!n,i'urati!n ,ies !n y!ur )!r6space.
r.
Cic6 !n the Ser)ices in6 t! vie) the avaia+e services. 5he ne)y created c!nverter *e+ service )i +e sh!)n there.
s. Cic6 !n !n th the !on)erter Ser)ice in6 t! dispay the )sd URL !, the ne)y created *e+ service. C!py the URL.
t.
N!) )e? )e? 'ener 'enerate ate the cie cient nt ,!r the ne)y ne)y create created d service service +y re,erri re,errin' n' the )sd )sd 'enerated +y the A0is- Server. Open ;ie Ne) Other... *e+ *e+ Services *e+ *e+ ServiceCient
u. Paste the URL URL that )as )as c!pied earier int! the service de,initi!n de,initi!n ,ied. ,ied.
v. Cic6 !n the !lient pro7ect hyperin6 and enter %xis9+STest!lient as the na(e !, the cient pr!9ect. Cic6 O.
Bac6 !n the *e+ Services Cient )i
Ne0t pa'e is the Cient C!n,i'urati!n C!n,i'ura ti!n Pa'e. Accept the de,auts and cic6 ;inish.
5he Cients stu+s )i +e 'enerated t! y!ur Dyna(ic *e+ pr!9ect %xis9+STest!lient .
N!) )e are '!in' t! )rite 1ava (ain pr!'ra( pr! 'ra( t! inv!6e the cient stu+. s tu+. I(p!rt the C!nverterCient.9ava ,ie t! the )!r6space int! the )tp pac6a'e in the src ,!der !, %xis9+STest!lient .
5hen seect the C!nverterCient ,ie8 ri'htcic6 and seect Run As 1ava Appicati!n. >ere?s )hat y!u 'et !n the server c!ns!e@
An!ther )ay t! test and inv!6e the service is t! seect enerate test case to test the ser)ice chec6 +!0 !n the A0is- Cient *e+ Service C!n,i'urati!n Pa'e )hen '!in' thr!u'h the *e+ Service Cient )i
I, that !pti!n is seected8 the A0is- e(itter )i 'enerate 1Unit testcases (atchin' the *SDL )e pr!vide t! the cient. 5hese 1Unit testcases )i +e 'enerated t! a ne)y added s!urce direct!ry t! the %xis9+STest!lient pr!9ect caed test.
Ne0t thin' )e need t! d! is t! insert in sert the test case )ith the vaid v aid inputs as the *e+ *e+ service (eth!d ar'u(ents. In this case8 et?s test the C!nverterC!nverterSOAP$$P!rtQhttp5est.9ava +y pr!vide vaues ,!r Cesius and ;arenheit ,!r the te(perature c!nversi!n. As an e0a(pe8 repace the 'enerated 5ODO state(ent in each test (eth!d t! ,i in the data )ith vaues as@ test,arenheit5!Cesius test,arenheit5!Cesius ,arenheit5!Cesius=.set;arenheit-$- ,arenheit5!Cesius=.set;arenheit-$- testStart,arenheit5!Cesius testStart,arenheit5!Cesius ,arenheit5!Cesius=.set;arenheit-$- ,arenheit5!Cesius=.set;arenheit-$- testcesius5!;arenheit testcesius5!;arenheit cesius5!;arenheit$#.setCesius$## cesius5!;arenheit$#.setCesius$## testStartcesius5!;arenheit testStartcesius5!;arenheit cesius5!;arenheit$#.setCesius$## cesius5!;arenheit$#.setCesius$##
>ere the testcases )ere 'enerated t! test +!th the synchr!n!us and asynchr!n!us cients. ). A,ter that8 seect seect the testcase8 ri'htcic68 ri'htcic68 seect Run As 1Unit 5est. 5est. !u !u )i +e a+e t! run the unit test success,uy inv!6in' the *e+ service.
5he *e+ Service )i
!u can ch!!se *e+ *e+ Services 5est 5est )ith *e+ Services E0p!rer t! test the service.
!u can ch!!se *e+ Services Pu+ish *SDL ,ie t! pu+ish the service t! a pu+ic UDDI re'istry re'istr y.
RES#$T'
5hus the deve!p(ent !, a ne) OGSAc!(piant )e+ service )as e0ecuted success,uy.
#sing %pache %xis 0e)elop a ri0 Ser)ice Ex No' =
D%TE'
"&-E!TIE'
5! deve!p a Grid Service usin' Apache A0is.
PR"!ED#RE'
!u )i need t! d!)n!ad and insta the ,!!)in' s!,t)are@ $. 1ava - SD v$.".$8 [email protected]!(29-se2$.".$2d!)n @229ava.sun.c!(29-se2$ .".$2d!)n!ad.ht( !ad.ht( -. Apache 5!(cat v".$-" [email protected].!r'2+uids29a6artat!(cat".#2reease2v".$.-"2+in29a6arta t!(cat".$.-".e0e.. t!(cat".$.-".e0e /. HML Security v$.#."8 http@22))).apache.!r'2dist20(2security29avai+rary20(security +in$.#.".
(. -a)a 9 SDK S
S
Run the d!)n!aded e0ecuta+e 9-sd6$Q"Q$)ind!)si4=7.e0e )hich )i insta SD in C@T9-sd6$.".$. Set the 1AAQ>OME envir!n(ent varia+e t! p!int t! this direct!ry as ,!!)s@
S
S
S
S
S
Cic6 !n S5AR5 S5AR5CON CON5ROL 5ROL PANELS PANELSS5EM S5EM Cic6 !n the Advanced ta+ Cic6 !n the Envir!n(ent aria+es +utt!n Cic6 !n the Ne) +utt!n in the user varia+e secti!n and enter the detais Add the 1ava +inaries t! y!ur PA5> varia+e in the sa(e )ay +y settin' a user varia+e caed PA5> )ith the vaue va ue JVPA5>VC@T9-sd6$.".$T+inK JVPA5>VC@T9-sd6$.".$T+inK
9. %pache Tomcat S
Run the d!)n!aded e0ecuta+e 9a6artat!(cat".$.-".e0e8 and assu(e the instaati!n direct!ry is C@T9a6artat!(cat".$.-".
S
Edit Edit C@T 9a6arta 9a6artat!( t!(cat cat".$ ".$.-"T .-"Tc!n, c!n,Tt! Tt!(cat (catuse users.0 rs.0( ( and create create an Jad(inK Jad(inK and J(ana'erK r!e as )e as a user )ith +!th r!es. 5he c!ntents !, the ,ie sh!ud +e si(iar t!@ W0( versi!nX?$.#? enc!din'X?ut,=?
Wt!(catusers Wr!e r!ena(eXY(ana'erY2 Wr!e r!ena(eXYad(inY2 Wuser userna(eXY(yuserY pass)!rdXY(ypassY r!esXYad(in8(ana'erY2 r!esXYad(in8(ana' erY2 W2t!(catusers S
Start 5!(cat +y runnin' C@T 9a6artat!(cat".$.-"T+inTstartup.+at and test it +y +r!)sin' http@22!cah!st@=#=#2
S
St!p 5!(cat +y runnin' C@T 9a6artat!(cat".$.-"T+inTshutd!)n.+at.
=. XM$ Security S
D!)n!ad and un
S
S
C!py 0(sec.9ar t! C@Ta0is$Q$Ti+T Setup y!ur CLASSPA CLASSPA5> envir!n(ent varia+e t! incudin' the ,!!)in'@ C@Ta0is$Q$Ti+T0(sec.9ar
5. %pache %xis S
S
Un
S
Setup y!ur CLASSPA CLASSPA5> envir!n(ent varia+e va ria+e t! incudin' in cudin' the ,!!)in'@ , !!)in'@ ! 5he current )!r6in' direct!ry ! A the AHIS 9ar ,ies as ,!und in C@Ta0is$Q$Ti+
C@T9a6artat!(cat".$.-"Tc!((!nTi+Tservet.9ar S
!ur CLASSPA CLASSPA5> sh!ud sh!u d there,!re th ere,!re !!6 s!(ethin' i6e@ C@Ta0is$Q$Ti+Ta0is.9ar C@Ta0is $Q$Ti+Ta0isant.9ar
C@Ta0is$Q$Ti+Tc!((!nsdisc!very.9ar C@Ta0is$Q$Ti+Tc!((!ns!''in'.9ar C@Ta0is$Q$Ti+T9a0rpc.9ar
C@Ta0is$Q$Ti+T!'"9$.-.=.9ar C@Ta0is$Q$Ti+Tsaa9.9ar
C@Ta0is$Q$Ti+T)sd"9.9ar C@Ta0is$Q$Ti+T0ercesI(p.9ar C@Ta0is$Q$Ti+T0(ParserAPIs.9ar
S
C@T9a6artat!(cat".$.-"Tc!((!nTi+Tservet.9ar
C@Ta0is$Q$Ti+T0(sec.9ar
N!) te 5!(cat 5!(cat a+!ut y!ur A0is )e+ appicati!n +y creatin' cr eatin' the ,ie C@T9a6arta t!(cat".$.-"T)e+appsTa0is.0( )ith the ,!!)in' c!ntent@ WC!nte0t pathXY2a0isY d!cBaseXYC@Ta0is$Q$T)e+appsTa0isY d!cBaseXYC@Ta0is$Q $T)e+appsTa0isY de+u'XY#Y privie'edXYtrueY WL!''ercassNa(eXY!r'.apache.cataina.!''er.;ieL!''erYpre,i0XYa0isQ!' .Y su,,i0XY.t0tY ti(esta(pXY,aseY2
. Deploy a Sample +e, ser)ice pac>age0 *ithin %xis installations
Dep!y !ne !, the sa(pe *e+ Services t! test the syste( and t! create the C@Ta0is $Q$T)e+a $Q$T)e+appsT ppsTa0is a0isT*E T*EBIN BIN;Tse ;Tserver rverc!n c!n,i'. ,i'.)sd )sdd d ,ie. ,ie. ;r!( ;r!( C@Ta0is C@Ta0is$Q$ $Q$ issue issue the c!((and !n !ne ine@ 9ava !r'.apache.a0is.cient.Ad(inCient http@22!cah!st@=#=#2a0is2services2Ad(inService2sa(pes2st!c62dep!y.)sdd 5his sh!ud return the ,!!)in'@ .
.
Pr!cessin' ,ie sa(pes2st!c62dep!y.)sdd WAd(inD!ne pr!cessin'W2Ad(in
RES#$T'
5hus the deve!p(ent !, a Grid Service usin' Apache A0is is e0ecuted success,uy.
De)elop applications using -a)a or !C! ri0 %PIs Ex No' 5
D%TE'
"&-E!TIE'
5! deve!p an appicati!ns usin' 1ava !r C2C33 Grid APIs.
S%MP$E !"DE'
i(p!rt A'ent5ea()!r6.Atea(.Z i(p!rt MP1.Z pu+ic cass UserPr!'Atea( e0tends e0 tends Atea(Pr!' [ private int phase pu+ic UserPr!'Atea( Atea( ! [\ pu+ic UserPr!'Atea( [\ 22 rea c!nst pu+ic UserPr!'Atea( Strin'F ar's [ phase X #R \ 22 phase rec!very private v!id userRec!very userRec!v ery [ phase X atea(.'etSnapsh!tId atea(.'etSnapsh!tId \ private v!id c!(pute [ ,!r phase X # phase W $# phase33 [ try [ 5hread.current5hread .seep $### \ catchInterruptedE0cepti!n e [ \ atea(.ta6eSnapsh!t phase Syste(.!ut.printn YUserPr!'Atea( at ran6 Y 3 MP1.COMMQ*ORLD.Ran6 3 Y @ t!!6 a snapsh!t Y 3 phase \ \ pu+ic static v!id (ain Strin'F ar's [ Syste(.!ut.printn YUserPr!'Atea(@ '!t startedY MP1.Init ar's8 atea( UserPr!'Atea( pr!'ra( X nu 22 5i(er ti(er X ne) 5i(er i, atea(.isResu(ed [ pr!'ra( X UserPr!'Atea( atea(.retrieveL!caar atea(.retrieveL!caar Ypr!'ra(Y Ypr!' ra(Y pr!'ra(.userRec!very
\ ese [ pr!'ra( X ne) UserPr!'Atea( ar's atea(.re'isterL!caar Ypr!'ra(Y8 pr!'ra( \ pr!'ra(.c!(pute MP1.;inaiet sample co0e : *ithin some function ,o0y
i(p!rt A'ent5ea()!r6.Atea(.Grid5 A'ent5ea()!r6.Atea(.Grid5cp.Z cp.Z private ,ina int p!rt p! rt X -### private GridS!c6et s!c6et s!c6 et private GridServerS!c6et server private InputStrea( input private OutputStrea( !utput ,!r int i X start i W start 3 trans i33 [ try [ !utput.)rite i V $-= \ catch IOE0cepti!n e [ \ Syste(.!ut.printn YS!c6ets )ith Y 3 (yRan6 3 Y@ Y 3 Y !utputY 3 i 3 YFXY 3 i V $-= \ ,!r int i X start i W start 3 trans i33 [ try [ Syste(.!ut.printn YS!c6ets )ith Y 3 (yRan6 3 Y@ Y 3 Y inputY 3 i 3 YFXY 3 input.read \ catch IOE0cepti!n e [ \\ MPI sample co0e
i(p!rt A'ent5ea()!r6.Atea(.Z i(p!rt MPI.Z pu+ic cass UserPr!'Atea( e0tends e0 tends Atea(Pr!' [ 22 appicati!n +!dy private v!id c!(pute [ \
pu+ic static v!id (ain Strin'F ar's [ MP1.Init ar's8 atea( pr!'ra(.c!(pute MP1.;inai
]2+in2sh r( , Z.cass 9avac casspath MP1.9ar@Atea(.9ar@. MP1.9ar@Atea (.9ar@. Z.9ava ] 9ar cv, Grid1NI.9ar Z.cass 9ar cv, Grid1NI.9ar Z.cass 9avah 9ni 1ava5!Cpp '33 rdyna(ic 1ava5!Cpp.cpp ! Qi+1ava5!Cpp.s!Q shared d '33 shared ! Qi+>e!*!rd.s!Q Qi+>e!*!rd.s!Q Grid1NIQi+rary.cpp Grid1NIQi+ rary.cpp >e!*!rd.cpp !C! MPI sample co0e : Hello*orl0.cpp
]incude Wi!strea(.h usin' na(espace std typede, int MPIQReuest8 MPIQStatus8 MPIQC!(( e0tern v!id ta6eSnapsh!tint ar'c e0tern int MPIQInitintZ ar'c8 charZZZ ar'v e0tern v!id MPIQ;inaie!*!rd.cppFCain' Ran6 and Sie!*!rd.cppFRan6 X Y WW ran6 WW end c!ut WW Y>e!*!rd.cppFSi
5hus the deve!p(ent !, appicati!ns usin' 1ava !r C2C33 Grid APIs is e0ecuted success,uy
Ex No'
De)elop secure0 applications using ,asic security mechanisms a)aila,le in lo,us Tool>it
D%TE'
"&-E!TIE'
5! deve!p a secured appicati!ns usin' +asic security (echanis(s avaia+e in G!+us.
PR"!ED#RE'
5he G!+us 5!!6it?s 5!!6it?s Authenticati!n Authenticati!n and Auth!ri
s!,t)are deve!p(ent 6its SDs pr!vide pr!'ra((in' i+raries8 1ava casses8 and essentia t!!s ,!r a PI8 certi,icate+ased authenticati!n syste( )ith sin'e si'n!n and dee'ati!n ,eatures8 in either *e+ Services !r n!n*e+ Services ,ra(e)!r6s. YDee'ati!nY YDee'ati!nY (eans that !nce s!(e!ne accesses a re(!te syste(8 he can 'ive the re(!te syste( per(issi!n t! use his credentias t! access !thers syste(s !n his +eha,.
+E& SERI!ES %#THENTI!%TI"N %ND %#TH"RI%TI"N :
A *e+ services i(pe(entati!n !, the Grid Security In,rastructure GSI8 c!ntainin' the c!re i+raries and t!!s needed t! secure appicati!ns usin' GSI (echanis(s. 5he Grid is a ter( c!((!ny used t! descri+e a distri+uted c!(putin' in,rastructure )hich )i a!) Yc!!rdinated res!urce sharin' and pr!+e( s!vin' in dyna(ic8 (utiinstituti!na virtua !r'ani55P >55P transp! transp!rt rt pr!t!c pr!t!c! ! and the HML HML ,a(i ,a(iy y !, standa standards rds.. In particuar8 *e+ Services use the SOAP (essa'in' standard ,!r c!((unicati!n +et)een service and reuest!r. reuest!r. 5hey sh!ud +e se,descri+in'8 se,descri+in'8 se,c!ntained se,c!ntained and (!duar present a pat,!r( and i(pe(entati!n neutra c!nnecti!n ayer and +e +ased !n !pen standards ,!r descripti!n8 disc!very and inv!cati!n. 5he Grid Security Security In,rastructure GSI is +ased !n the Generic Security Services API GSSAPI and uses an e0tensi!n t! H4#% certi,icates t! pr!vide a (echanis( t! authenticate
su+9ects and auth!rise res!urces. It a!)s users t! +ene,it ,r!( the ease !, use !, a sin'e si'n!n (echanis( +y usin' dee'ated credentias8 and ti(ei(ited pr!0y certi,icates. GSI is used as the security in,rastructure ,!r the G!+us 5!!6it. Recent Recenty y8 a ne) pr!p!sa pr!p!sa ,!r an Open Open Grid Services Services Architectu Architecture re OGSA OGSA )as ann!unced )hich (arries the Grid and *e+ Services t! create a ne) Grid Services (!de. One pr!+e(8 )hich has n!t yet +een e0picity addressed8 is that !, security. A p!ssi+e s!uti!n is t! use a suita+y secure transp!rt +indin'8 e.'. 5LS8 and e0tend it t! inc!rp!rate appr!priate supp!rt ,!r pr!0y credentias. It )!ud +e use,u t! test !ut s!(e !, the principes !, Grid Services usin' the currenty avaia+e ,ra(e)!r6s and t!!s ,!r deve!pin' *e+ Service Services. s. Un,!rtun Un,!rtunate atey y88 n! standard standardss current currenty y e0ist e0ist ,!r i(pe( i(pe(ente ented d pr!0y pr!0y credent credentia ia supp! sup p!rt rt t! pr!vi pr!vide de authen authenti ticat cated ed c!((u c!((unic nicati ati!n !n +et)e +et)een en )e+ )e+ servi service ces. s. A nu( nu(+e +err !, HML2* HML2*e+ e+ Service Servicess securit security y standard standardss are current currenty y in deve!p( deve!p(ent8 ent8 e.'. HML Di'ita Di'ita Si'natures8 SAML8 HMS8 HACML8 +ut the re(ainder !, this d!cu(ent descri+es an appr!ach pr!p!sed +y ANL t! use GSI !ver an SSL in6. A 'eneric 1!+ Su+(issi!n envir!n(ent8 GAP ena+es researchers and scientists t! e0ecute their appicati!ns !n Grid ,r!( a c!nventi!na )e+ +r!)ser. B!th Seuentia and Parae Parae 9!+s 9!+s can can +e su+( su+(itte itted d t! GARUDA GARUDA Grid thr!u'h thr!u'h P!rta. P!rta. It pr!vides a )e+ inter,ace ,!r vie)in' the res!urces8 and and ,!r su+(ittin' and (!nit!rin' (!nit!rin' 9!+s.
Pre8reuisites for using %P
P!rta users need t! set the ,!!)in' in their 2.+ashrc ,ie. e0p!rt GLOBUSQLOCA5IONX2!pt2asvi9a2GLOBUS".#.:2 GLOBUSQLOCA5IONX2!pt2asvi9a2GLOBUS".#.:2 s!urce 2!pt2asvi9a2GLOBUS".#.:2etc2'!+ususerenv.sh e0p!rt PA5>X2usr2!ca29d6$.7.#Q$ 5>X2usr2!ca29d6$.7.#Q$#2+in@ #2+in@ G*QLOCA5ION2+in@2!pt2'arudaresv G*QLOCA5ION2+in@2!pt2'arudaresv2+in@2!pt2v!(sQcien 2+in@2!pt2v!(sQcien t2+in@_PA t2+in@_PA5> e0p!rt LDQLIBRARQPA5>X_LDQLIBRARQPA5>@2!pt2v!(sQcient2i+@ %ccessing %P
5ype http@22$%-.$7=.7#."#2GridP!rta$./2 t! access the P!rta thr!u'h GARUDA
Net)!r6 !r http@22-#/.-##./7.-/72GridP!rta$./ t! access the P!rta thr!u'h Internet in the address +ar !, the )e+ +r!)ser t! inv!6e the P!rta. It is pre,era+e t! access the P!rta thr!u'h GARUDA Net)!r68 since it is (uch ,aster than the Internet. In !rder t! access the ,aciities !, Grid P!rta such as 1!+ Su+(issi!n8 1!+ Status trac6in'8 St!rin'Up St!rin'Up!adin' !adin' !, E0ecuta+es and ie) Output2Err!r Output2Err!r data8 the user has t! !'in int! the P!rta usin' the User?s L!'in ;!r( in the >!(e pa'e !, the P!rta. a Ne) users users are reuired t! cic6 Si'n Si'n up in the User L!'in ;!r(8 ;!r(8 )hich )hich eads the( t!
h!(e
pa'e
!,
Indian Indian
Grid
Certi,ica Certi,icati!n ti!n
Auth!rity Auth!rity
IGCA
http@22ca.'arudaind ia .i .i n2. Cic6 !n Reuest Certi,icate and acuire the reuired user2h!st certi,icates8 certi, icates8 detais de tais are pr!vided in i n IGCA secti!n. + Re'istered users are reuired t! pr!vide User Id and Pass)!rd ,!r !''in' int! the P!rta and access vari!us ,aciities. -o, Management
User can su+(it their 9!+8 (!nit!r the status and vie) !utput ,ies usin' the 1!+ Mana'e(ent inter,aces. inter,aces. 5ypes !, !, 9!+ su+(issi!n su+(issi!n Basic and
Advanced and 1!+
in,!r(ati!n are c!vered under under this secti!n. secti!n. &asic -o, Su,mission
5his inter,ace can +e used t! su+(it seuentia as )e as parae 9!+s. 5he user sh!ud pr!vide the th e ,!!)in' in,!r(ati!n@ in,! r(ati!n@ $. Opti!na Opti!na 1!+ 1!+ Na(e User User can pr!vide pr!vide a suita+e suita+e aias aias na(e ,!r their their 9!+. 9!+. -. 5y 5ype pe !, 1!+ user user )ant )ant t! e0ec e0ecute ute88 /. Operati Operatin' n' Syste( Syste( ` Reuir Reuired ed ,!r ,!r their their 1!+8 1!+8 ". ?>ave ?>ave y!u reserved the Res!urc Res!urces? es? An !pti!na !pti!na para(eter para(eter c!ntain c!ntainss the Reservati! Reservati!n n Id?s that can +e used ,!r 9!+ su+(issi!n instead !, ch!!sin' the the Operatin' Syste(2Pr!cess!r Syste(2Pr!cess!r para(eter. 4. N!. !, pr!cesses pr!cesses reuired reuired ,!r the 9!+ 5his para(et para(eter er is !ny ,!r the parae parae appicati!ns that reuire (!re than !ne CPU. 7. C!rresp!ndin' C!rresp!ndin' E0ecuta+es E0ecuta+es ` up!aded up!aded ,r!( ,r!( either either !ca !ca !r re(!te (achine8 :. Input ,ie8 ,ie8 i, reuired reuired 5he e0ecuta+e e0ecuta+e and the input ,ie ,ie can either +e up!aded ,r!( the !ca (achine !r can +e seected ,r!( the Re(!te ;ie List8 i, it is avaia+e in the Su+(it N!de =. S5DIN S5DIN Reuired Reuired )hen )hen the user user )ants t! t! pr!vide pr!vide any inputs inputs t! the appicat appicati!n i!n
durin' the runti(e. %. Opti! Opti!na na E0ecu E0ecuti! ti!n n 5i(e >ere >ere the E0ecu E0ecuti! ti!n n 5i(e is the antici anticipat pated ed 9!+ c!(peti!n ti(e. $#. Any C!((and Line ar'u(ents ar'u(ents !r Envir!n(ent aria+es8 aria+es8 i, reuired. $$. User Speci,ic Output2 Output2 Err!r ,ies ,ies I, the appicati!n 'enerates !utput2err!r !utput2err!r ,ies !ther than standard !utput2err!r !utput2err!r ,ies and its entries sh!ud +e separated +y c!((a?s !r sin'e e(pty space in case !, (utipe ,ies. A th!se ,ieds (ar6ed )ith Z are (andat!ry ,ieds and sh!ud +e ,ied +e,!re su+(ittin' a 9!+. By cic6in' !n su+(it +utt!n8 the p!rta su+(its the 9!+ t! Grid *ay Meta Scheduer8 )hich then schedues the 9!+ ,!r e0ecuti!n e0ecuti!n and returns the 1!+ Id. 5he 1!+ Id has t! +e n!ted ,!r ,uture re,erence t! this 9!+. In the event !, unsuccess,u su+(issi!n8 the c!rresp!ndin' err!r (essa'e is dispayed. A th!se ,ieds (ar6ed )ith Z are (andat!ry ,ieds and sh!ud +e ,ied +e,!re su+(ittin' a 9!+. By cic6in' !n su+(it +utt!n8 the p!rta su+(its the 9!+ t! Grid*ay Meta Scheduer8 )hich then schedues the 9!+ ,!r e0ecuti!n e0ecuti!n and returns the 1!+ Id. 5he 1!+ Id has t! +e n!ted ,!r ,uture re,erence t! this 9!+. %0)ance0 -o, Su,mission
5his inter,ace is pr!vided ,!r the user t! su+(it their Seuentia and Parae 1!+s. 5he di,,erence di,,erence ,r!( Basic 9!+ su+(issi!n +ein'@ it is usin' G5" *e+ Services Services
c!(p!nents ,!r su+(ittin' 9!+s t! the Grid instead !, Grid)ay as scheduer. 5he user is pr!vided )ith t)! (!des in this inter,ace@ $. De,aut De,aut (!de P!rta P!rta create createss the the HML ,ie ,!r the user. user. -. Sec!nd (!de8 rec!((ended ,!r advanced advanced users 5he 5he user can pr!vide their!)n HML ,ie as the e0ecuta+e8 pr!vided the reuired ,ies are avaia+e in the su+(it n!de. 5he user can vie) the status !, the 9!+ su+(itted thr!u'h P!rta and the !utput ,ie !, the 9!+ +y speci,yin' the 1!+ Id. 5he !pti!n ,!r d!)n!adin' the Output2 Err!r ,ie is as! pr!vided8 a,ter the 9!+ e0ecuti!n. 5! cance any !, the ueued 9!+s8 the user has t! seect the 9!+ and cic6 Cance 1!+ +utt!n8 ,!!)in' )hich the ac6n!)ed'(ent ,!r the 9!+ canceed canc eed is pr!vided. pr!vide d.
-o, Info
5he user can vie) the status !, the 9!+ su+(itted thr!u'h P!rta and the !utput ,ie !, the 9!+ +y speci,yin' the 1!+ Id. 5he !pti!n ,!r d!)n!adin' the Output2 Err!r ,ie is as! pr!vided8 a,ter the 9!+ e0ecuti!n. 5! cance any !, the ueued 9!+s8 the user has t! seect the 9!+ and cic6 Cance 1!+ +utt!n8 ,!!)in' )hich the ac6n!)ed'(ent ,!r the 9!+ canceed canc eed is pr!vided. pr!vide d.
Resources
5he Grid*ay (etascheduer pr!vides the ,!!)in' in,!r(ati!n N!de Na(e8
>ead N!de8 OS8 ARC>8 L!ad Avera'e8 Status8 C!n,i'ured Pr!cess and Avaia+e Pr!cess. 5his in,!r(ati!n aids user t! seect a suita+e custer and reserve the( in advance ,!r 9!+ su+(issi!n.
Steps for Reser)ation of Resources
$. Chec6 the avaia+e ,ree res!urces res!urces )ith vaid para(eters para(eters Start Start 5i(e and and End 5i(e ` durati!n ,!r )hich the res!urce needs t! +e reserved. 5he input ,ieds N!. !, CPUs and OS entries are !pti!na. E0a(pe@ startti(eX -##%#"#- $:@#7@4/ endti(eX-##%#"#- $%@#:@$# N!. !, CPUsX- OS NAMEXLinu0 -. Ch!!se th the Availa&le 'rocess reuired ,!r the 9!+. E0a(pe@ Avaia+e Pr!cs X " /. Seect Seect the reuire reuired d res!urce res!urce ,r!( ,r!( the avaia+ avaia+e e ist !, res!ur res!urces. ces. ". B!!6 B!!6 the res!urces res!urces ,!r ,!r reservin' reservin' a res!ur res!urce ce ,!r the reuest reuested ed peri!d peri!d !, ti(e and and pr!cess. 4. 5he reserved reserved res!urce res!urcess can can +e (!di, (!di,ied2 ied2 canceed. canceed. 7. Once the reservati!n pr!cess is success,uy success,uy c!(peted8 c!(peted8 the Reservati!n Id is dispayed dispayed and is (ade avaia+e in the Basic 1!+ Su+(issi!n pa'e. File ,ro*ser
;!r the !''edin user8 the ;ie Br!)ser ists ists ,ies8 such as the up!aded e0ecuta+es and Input2Output2Err!r Input2Output2Err!r ,ies8 a!n' )ith their si
%ccounting
5his (!due pr!vides Acc!untin' in,!r(ati!n !, the 9!+s that are su+(itted t! GARUDA8 such as n!. !, 9!+s su+(itted8 and syste( para(eters such as Me(!ry usa'e8 irtua (e(!ry8 *a 5i(e8 and CPU ti(e. Last !ne (!nth data is dispayed +y de,aut.
MyProxy
MyPr!0y a!)s user t! up!ad their G!+us Certi,icates int! Mypr!0y Server and the sa(e can +e used ,!r initiaiOME2.'!+us direct!ry usin' )inscp2scp.
MyProxy Init
By de,aut8 the YMypr!0y InitY !pti!n is ena+ed ,!r the user. Up!ad pr!0y +y enterin' enterin' vaid inputs inputs User na(e8 Gridpr! Gridpr!0y 0y Passphra Passphrase8 se8 User certi,ica certi,icate te ,ie usercert.pe(8 User 6ey ,ie user6ey.pe( and Pr!0y i,e ti(e $7= h!urs is the de,aut vaue. MyProxyet
Grid pr!0y )i +e initiai
5he irtua Or'ani
5he user can as! reuest ,!r a ne) O +y usin' YReuest ,!r OY in6. OMS pr!0y initiai
L!'in t! 'rid,s$%-.$7=.7#."# Up!ad y!ur IGCA user certi,icates Initiai
p)fs9 1(J9.9B.(.;(2 n!de sh!ud +e used t! 9ust test a the avaia+e SRM cient
inter,aces i6 e St!RM8 DP(8 BestMan. gri0fs 1(O9.(;.B.5B2 n!de sh!ud8 i, the user )ishes t! use GSRM st!ra'e
,!r 9!+ e0ecuti!n. Users Users can d!)n!ad2Up!ad input2!utput input2!utput ,ies int! GSRM )hie su+(ittin' 9!+s ,r!( 'rid,s. ;!!)in' %ccess mechanisms are avaia+e at a+!ve (enti!ned n!des t! access GSRM@ $. gri0fs1(O9.(;.B.5B2 @ 'rid,s is the Ban'a!re GARUDA head n!de. GSRM services can +e accessed ,r!( here usin' St!RM c!((and ine inter,ace. I, the user )ants t! use the cientSRM St!RM Cients ,r!( 'rid,s (achine Create a vaid user pr!0y usin' 'ridpr!0yinit Set the env varia+e ,!r G!+us !cati!n path e0p!rt GLOBUSQLO GLOBUSQLOCA CA5IONX 5IONX GLOBUSQLO GLOBUSQLOCA CA5ION@ 5ION@2usr2!c 2usr2!ca2GAR a2GARUDA2G UDA2GLOBUS LOBUS ".#.:2 e0p!rt PA5>X_PA5>@2!pt2'sr(cient2sr(v-st!r(2+in e0p!rt LDQLIBRARQPA5>X_LDQLIBRARQPA5>@2!pt2'sr( cient2c'siQs!ap2i+ Run the cientSRM c!((and -. p)fs9 1(J9.9B.(.;(2' pv,s- is the GSRM testin' n!de )ith the ,!!)in' cient inter,aces instaed. Best(an 1ava APIs DPM C APIs /. SRM +e, !lient is accessi+e ,r!( any !, the user (achines reacha+e t! GSRM server
0n#4.cts,.cdac.!r'.in8 0n#4.cts,.c dac.!r'.in8 usin' URL https'CCxnB.ctsf.c0ac.org.inC
SRM !lient Interfaces
StoRM !omman0 $ine !lient
$. St!RM St!RM c!((an c!((and d ine ine cie cient nt ,!r(a ,!r(at@ t@ cientSRM WreuestNa(e WreuestOpti!ns -. 5! 'et 'et hep hep ,!r cie cientS ntSRM RM c!(( c!((and ands@ s@ cientSRM `h /. C!((an C!((and d t! pin' pin' t! GSRM GSRM serve server@ r@ cientSRM pin' e WGSRM end p!int
&estman !omman0 $ine !lients
$. C!((an C!((and d t! pin' pin' t! GSRM GSRM serve server r sr(pin' `serviceur http'@220n#4.cts,.cdac.!r'.in@=""72dp(2cts,.cdac.!r'.in2h!(e2'aruda -. Up!a Up!ad d ,ie ,ie t! GSRM GSRM serve server r sr(c!py Wsrc ur Wtar'et ur Wservice ur Pre8reuisites for using S"% compiler
$. 1ava Run Run 5i(e 5i(e Envir Envir!n( !n(ent ent 1D 1D$.7 $.73 3 -. *e+ Br!)ser Br!)ser )ith 1ava )e+ start supp!rt supp!rt !ompiler #I
5he users are reuired t! adhere t! ,!!)in' direct!ry direct!r y structure. Appicati!n Parent Dir src28+in28i+28incude2 (2 $ogin
5his (eth!d is ,!r !''in' in t! the GARUDA. Inputs
user na(e
MyPr!0y User Na(e
pass)!rd
MyPr!0y Pass)!rd Pas s)!rd
i,e ti(e
Indicates h!) !n' is the pr!0y?s i,e ti(e
"utput
Pr!0y strin' strin'
Pr!0y issued +y the My pr!0y server
L!'in status
Indicates the status !, the !perati!n
Last L!'in 5i(e
Gives )hen this user )as ast !''ed in
Current L!'in 5i(e
Gives users !''in' in ti(e
92 uploa0Proxy
5his (eth!d up!ads a pr!0y th at is 'enerated usin' !ther t! !s8 t! the MyPr!0y Server. Inputs
user na(e
MyPr!0y User Na(e
pass)!rd
MyPr!0y Pass)!rd Pas s)!rd
pr!0yBytes
E0istin' pr!0y pr! 0y ,ie is 'iven as +yte +y te array
"utput
up!adStatus up!adStatu s
Indicates the status !, the !perati!n
=2 store!re0ential
5his (eth!d is used ,!r up!adin' the credentias that is the PCS$- certi,icate directy t! the MyPr!0y Server. Server. It )i c!nvert the PCS$PCS$- t! certi,icate certi,icate and st!res in server ,!r users t! d!)n!ad the the pr!0y unti it e0pires. e0pires. Inputs
user na(e
MyPr!0y User Na(e
pass)!rd
MyPr!0y Pass)!rd Pas s)!rd
p$-Bytes
PCS$- ,ie as +yte +yt e array
"utput
st!reStatus
Indicates the status !, the !perati!n
RES#$T'
5hus the deve!p(ent !, secured appicati!ns usin' +asic security (echanis(s avaia+e in G!+us is e0ecuted success,uy.
Ex No'
De)elop a ri0 portal *here user can su,mit a 7o, 7 o, an0 get the result. Implement it *ith an0 *ithout R%M concept
D%TE'
"&-E!TIE'
5! deve!p a Grid p!rta8 )here user can su+(it a 9!+ and 'et the resut and t! i(pe(ent it )ith and )ith!ut GRAM c!ncept.
PR"!ED#RE'
$. Openin' the )!r6,!) edit!r 5he edit!r is a 1ava *e+start appicati!n d!)n!ad and instaati!n is !ny a cic6.
-. 1ava *e+start appicati!n D!)n!ad and insta
/. 1!+ pr!perty )ind!)@
". 5he in,!r(ati!n syste( can uery EGEE and G!+us in,!r(ati!n syste(s
4. List !, avaia+e 'rids
7. C!(putin' res!urces !, such a 'rid
:. Br!6er res!urce seecti!n Seect a Br!6er Grid ,!r the 9!+ Speci,y e0tra ran6s and reuire(ents ,!r the 9!+ in 1!+ descripti!n an'ua'e. 5he +r!6er )i ,ind the +est res!urce ,!r y!ur 9!+.
=. De,inin' input2!utput data ,!r 9!+s ;ie type Input@ reuired +y the 9!+ Output@ pr!duced +y the 9!+ ;ie !cati!n@ !ca@ (y des6t!p re(!te@ 'rid st!ra'e res!urce ;ie na(e@ Uniue na(e !, the ,ie ;ie st!ra'e type@ Per(anent@ ,ina resut !, *; !atie@ !ny used ,!r inter9!+ data trans,er
%. E0ecutin' )!r6,!)s )ith the PGrade p!rta D!)n!ad pr!0ies
$#. D!)n!adin' a pr!0y
$$. Ass!ciatin' the pr!0y )ith a 'rid
$-. Br!)sin' Pr!0ies
$/. *!r6,!) *!r6,!) e0ecuti!n e0 ecuti!n *!r6,!) p!rtet
$". O+servati!n +y the )!r6,!) p!rtet
$4. D!)n!adin' the resuts
RES#$T'
5hus the deve!p(ent !, a Grid p!rta8 )here user can su+(it a 9!+ and 'et the resut and t! i(pe(ent it )ith and )ith!ut GRAM is e0ecuted success,uy
Ex No' J
FIND PR"!ED#RE T" R#N THE IRT#%$ M%!HINE "F DIFFERENT !"NFI#R%TI"N. !HE!K H"+ M%NQ IRT#%$ M%!HINES !%N &E #TI$IED %T P%RTI!#$%R TIME
D%TE'
"&-E!TIE'
5! understand pr!cedure t! run the virtua (achine !, di,,erent c!n,i'urati!n. Chec6 h!) (any irtua (achines can +e utii
PR"!ED#RE' KM INST%$$TI"N !hec> that your !P# supports har0*are har0*are )irtualiation
5! run M8 y!u need a pr!cess!r that supp!rts hard)are virtuai
I, # it (eans that y!ur CPU d!esn?t supp!rt hard)are virtuaiernel 1if possi,le2
Runnin' a 7" +it 6erne !n the h!st !peratin' syste( is rec!((ended +ut n!t reuired. 5! serve (!re than -GB !, RAM ,!r y!ur Ms8y!u (ust use a 7"+it 6erne8 On a /-+it 6erne insta8 y!u? +e i(ited t! -GB RAM at (a0i(u( ,!r a 'iven M. To see if your processor is 58,it you can run this comman0' egrep 8c lm CprocCcpuinfo
I, # is printed8 it (eans that y!ur CPU is n!t 7"+it. I, $ !r hi'her8 it is. No* see if your running >ernel is 58,it 8 9ust issue the ,!!)in' c!((and@ uname :m x;<5 indicates a runnin' 7"+it 6erne. I, y!u use see i/=78 i"=78 i4=7 !r i7=78 y!u?re
runnin' a /-+it 6erne. _ ls Cli,Cmo0ulesC=.(.B8=B8 Cli,Cmo0ulesC=.(.B8=B8 genericC>ernelCarchCx;C> genericC>ernelCarchCx;C>)mC>)m )mC>)m
6v(a(d.6! @ AMD Pr!cess!r 6v(inte.6! @ Inte Pr!cess!r 6v(.6! @ erne !+9ect _ s 2dev26v( 2dev26v( Install Necessary Pac>ages
$. e(u6v( -. i+virt+in /. +rid'eutis ". virt(ana'er 4. e(usyste( _ su0o apt8get install emu8>)m _ su0o apt8get install li,)irt8,in _ su0o apt8get install ,ri0ge8utils _ su0o apt8get install )irt8manager _ su0o apt8get install emu8system 5! chec6 pac6a'e instaati!n Chec6 0p>g :lUgrep emu8>)m
_ virsh virsh] e0it erify Installation
!u can test i, y!ur insta has +een success,u )ith the ,!!)in' c!((and@ _ virsh c e(u@222syste( ist Id Na(e State I, !n the !ther hand y!u 'et s!(ethin' i6e this@ _ virsh c e(u@222syste( ist i+vir@ Re(!te err!r @ Per(issi!n denied err!r@ ,aied t! c!nnect t! the hypervis!r virsh ] versi!n virsh ] n!de in,! !reating MS _ virtinsta c!nnect e(u@222syste( n hardy r 4$- , hardy$.c!)- s $- c u+untu$".#".-servera(d7".is! vnc n!aut!c!ns!e !stype inu0 !svariant u+untu>ardy !r Open dis6 i(a'e Err!r _ sud! ch(!d ::: hardy.'c!)To run _ virtinsta c!nnect e(u@222syste( n hardy r 4$- , hardy$.c!)- s $- c
u+untu$".#".-servera(d7".is! vnc n!aut!c!ns!e !stype inu0 !svariant u+untu>ardy _ sud! virt(ana'er 5! Run irtua Machine
5! L!'in in Guest OS
T" R#N M IN "PEN ST%!K Step $ @ Under the Pr!9ect 5a+8 !lic> Instances. In the ri'ht side screen Cic6 Launch Instance. Step - @ In the detais8 Give the instance na(ee'. Instance$. Step /@ Cic6 Instance B!!t S!urce ist and ch!!se &oot from image Step "@ Cic6 I(a'e na(e ist and ch!!se the i(a'e currenty up!aded. Step 4@ Cic6 aunch. !ur M )i 'et created.
"PEN ST ST%!K %!K INST%$$%TI"N INST%$$%TI"N
_ ,ree `( its sh!)s the RAM Usa'e
_ sud! adduser stac6 pass)!rd@ stac6 _ sud! apt'et insta sud! ,!r updatin' SU _ sud! `I enterin' int! the r!!t user ] ech! Ystac6 ALLXALL NOPASS*D@ NOPASS*D@ ALLY ALLY 2etc2sud!ers it chan'es chan' es the stac6 s tac6 ,r!( n!r(a users t! super user ]cat 2etc2sud!ers 5! chec6 ast ine as stac6 ALL XALL NOPASS*ORD@ALL NOPASS*ORD@ALL ]e0it !'!ut ,r!( r!!t user log off your machine an0 login as stac6 user Prereuiste8 Prereuiste8 to install git open all the ports
( sudo apt-get install git ( git clone https)git*opens https)git*openstac+*or tac+*org,openstac+-dev g,openstac+-dev,devdstac+ ,devdstac+ _ s devstac6 sh!ud there _ cd devstac6 _ s it sh!)s a the ,ies in devstac6 _s ` !n'istin' the devstac6 ,ies _ nan! !ca.c!n, paste in !ca.c!n, !ca!carcFF ;LOA5INGQRANGEX$%-.$7=.$.--"2-: ;IHEDQRANGEX$#.$$.$-.#2-" ;IHEDQNE5*ORQSIbEX-47 ;LA5QIN5ER;ACEXeth# ADMINQPASS*ORDXr!!t DA5ABASEQPASS*ORDXr!!t RABBI5QPASS*ORDX r!!t SERICEQPASS*ORDX r!!t SERICEQ5OENX SERICEQ5OENX r!!t save and uit _ s ` sh!ud +e stac6 stac6 _ .2stac6.sh i, err!r nu(+er $=$ c!(es '! t! vi stac6.sh add ne0t t! UMAS #-;ORCEXyes _ p)d 2h!(e2stac62devstac6 Devstac6_ (v !ca.c!n, ..2 _ cd .. Stac6_ r( `r, devstac62 2!pt2stac62 _ git clone https)git*opens https)git*openstac+*or tac+*org,openstac+-dev g,openstac+-dev,devdstac+ ,devdstac+ `+ sta+e26i! _ cd devstac6 _ _ (v ..2!ca.c!n, . _ s ` !ca.c!n, _ .2stac6.sh _ nan! stac6.sh edit ](a6e sure un(as6 is sane Add ;ORCEXyes save and e0it _ .2unstac6.sh _ .2cean.sh Run De)Stac>'
_ .2stac6.sh
;ind User na(e8 Pass)!rd8 IP address
In +r!)ser 'ive y!ur (achines ip http:(()ip *ddress+
Re8Starting "penstac>
_ .2re9!in.sh _ ps e,'rep devstac6 it sh!)s a the pr!cesses runnin' End a the pr!cesses.
RES#$T' 5hus the pr!cedure t! run the virtua (achine !, di,,erent c!n,i'urati!n is e0ecuted success,uy.
Ex No' ;
FIND PR"!ED#RE P R"!ED#RE T" %TT%!H %TT%!H IRT#%$ &$"!K T" THE IRT#%$ M%!HINE %ND !HE!K +HETHER IT H"$DS THE D%T% EEN %FTER THE RE$E%SE "F THE IRT#%$ M%!HINE
D%TE'
"&-E!TIE'
5! )rite the pr!cedure t! attach virtua +!c6 t! the virtua (achine and chec6 )hether it h!ds the data even a,ter the reease !, the virtua (achine.
PR"!ED#RE'
!u(es are +!c6 st!ra'e st !ra'e devices that y!u attach t! instances t! ena+e persistent st!ra'e. !u can attach a v!u(e t! a runnin' instance !r detach a v!u(e and attach it t! an!ther instance at any ti(e. !u !u can as! create a snapsh!t ,r!( !r deete a v!u(e. Ony ad(inistrative users can create v!u(e types.
!reate a )olume $. L!' in t! the dash+!ard. -. Seect the appr!priate pr!9ect ,r!( the dr!p d!)n (enu at the t!p e,t. /. On the Pr!9ect ta+8 !pen the C!(pute ta+ and cic6 !u(es !u(es cate'!ry. ". Cic6 Create !u(e. In the dia!' +!0 that !pens8 enter !r seect the ,!!)in' vaues.
!u(e Na(e@ Speci,y a na(e ,!r the v!u(e. Descripti!n@ Opti!nay8 pr!vide a +rie, descripti!n ,!r the v!u(e.
!u(e S!urce@ Seect !ne !, the ,!!)in' !pti!ns@
N! s!urce8 e(pty v!u(e@ Creates an e(pty v!u(e. An e(pty v!u(e d!es n!t c!ntain a ,ie syste( !r a partiti!n ta+e.
I(a'e@ I, y!u ch!!se this !pti!n8 a ne) ,ied ,!r Use i(a'e as a s!urce dispays.
!u can seect the i(a'e ,r!( the ist.
!u(e@ I, y!u ch!!se this !pti!n8 a ne) ,ied ,!r Use v!u(e as a s!urce s !urce dispays. !u !u can seect the v!u(e ,r!( the ist. Opti!ns t! use a snapsh!t !r a v!u(e as the s!urce ,!r a v!u(e are dispayed !ny i, there are e0istin' snapsh!ts !r v!u(es. 5ype@@ Leave this ,ied +an6. 5ype
Si
On the 'roect ta+8 !pen the Compute ta+ and cic6 the .olumes cate'!ry. ". Seect the v!u(e and cic6 Manage Attachments. 4. Cic6 Detach .olume .olume and c!n,ir( y!ur chan'es.
A (essa'e indicates )hether the acti!n )as success,u.
!reate a snapshot from a )olume
$. L!' in t! the dash+!ard. -. Seect the appr!priate pr!9ect ,r!( the dr!p d!)n (enu at the t!p e,t. /. On the 'roe 'roect ct ta+8 !pen the Compute ta+ and cic6 .olumes cate'!ry. ". Seect a v!u(e ,r!( )hich t! create a snapsh!t. 4. In the Actions c!u(n8 cic6 Create Snapshot . 7. In the dia!' +!0 that !pens8 enter a snapsh!t na(e and a +rie, descripti!n. :. C!n,ir( y!ur chan'es. 5he dash+!ard sh!)s the ne) v!u(e snapsh!t in !u(e Snapsh!ts ta+.
E0it a )olume
$. L!' in t! the dash+!ard. -. Seect the appr!priate pr!9ect ,r!( the dr!p d!)n (enu at the t!p e,t. /. On the 'roe 'roect ct ta+8 !pen the Compute ta+ and cic6 .olumes cate'!ry. ". Seect the v!u(e that y!u )ant t! edit. 4. In the Actions c!u(n8 cic6 /dit .olume .olume. 7. In the /dit .olume .olume dia!' +!08 update the na(e and descripti!n !, the v!u(e. :. Cic6 /dit .olume .olume. Note
!u can e0tend a v!u(e +y usin' the /0tend .olume .olume !pti!n avaia+e in the More dr!pd!)n ist and enterin' the ne) vaue ,!r v!u(e si
Delete a )olume
*hen y!u deete an instance8 the data in its attached v!u(es is n!t deeted. $. L!' in t! the dash+!ard. -. Seect the appr!priate pr!9ect ,r!( the dr!p d!)n (enu at the t!p e,t. /. On the 'roe 'roect ct ta+8 !pen the Compute ta+ and cic6 .olumes cate'!ry. %ttach a )olume to an instance A,ter y!u create !ne !r (!re v!u(es8 y!u can attach the( t! instances. !u can attach a v!u(e t! !ne instance at a ti(e.
$. L!' in t! the dash+!ard. -. Seect the appr!priate pr!9ect ,r!( the dr!p d!)n (enu at the t!p e,t. /. On the 'roe 'roect ct ta+8 !pen the Compute ta+ and cic6 .olumes cate'!ry. ". Seect the v!u(e t! add t! an instance and cic6 Manage Attachments Attachments. 4. In the Manage .olume .olume Attachments Attachments dia!' +!08 seect an instance. 7. Enter the na(e !, the device ,r!( )hich the v!u(e is accessi+e +y the instance. Note Note'' 5he actua device na(e (i'ht di,,er ,r!( the v!u(e na(e +ecause !, hypervis!r
settin's :. Cic6 Attach .olume .olume. 5he dash+!ard sh!)s the instance t! )hich the v!u(e is n!) attached and the device na(e. !u can vie) the status !, a v!u(e in the !u(es ta+ !, the dash+!ard. 5he v!u(e is either Avaia+e Avaia+e !r InUse. N!) y!u can !' in t! the instance ins tance and (!unt8 ,!r(at8 and use us e the dis6. Detach a )olume from an instance
$. L!' in t! the dash+!ard. -. Seect the appr!priate pr!9ect ,r!( the dr!p d!)n (enu at the t!p e,t.
/. On the 'roe 'roect ct ta+8 !pen the Compute ta+ and cic6 .olumes cate'!ry. ". Seect the chec6 +!0es ,!r the v!u(es that y!u )ant t! deete. 4. Cic6 Delete .olumes .olumes and c!n,ir( y!ur ch!ice. A (essa'e indicates )hether the acti!n )as success,u.
RES#$T'
5hus the pr!cedure t! attach virtua +!c6 t! the virtua (achine and chec6 )hether it h!ds the data even a,ter the reease !, the virtua (achine is e0ecuted success,uy.
Ex No' O
INST%$$ % ! !"MPI$ER IN THE IRT#%$ I RT#%$ M%!HINE %ND EXE!#TE % S%MP$E PR"R%M
D%TE'
"&-E!TIE'
5! insta a C c!(pier and e0ecute a sa(pe pr!'ra( in the virtua (achine.
PR"!ED#RE' Step (' 5! !'in int! Guest OS in M
Step 9' 5! )rite and e0ecute y!ur !)n C Pr!'ra( in 'cc c!(pier.
Insta c c!(pier usin' c!((ands. _ apt'et insta 'cc
RES#$T'
5hus the instaati!n !, a C c!(pier and e0ecuti!n !, a C pr!'ra( in the virtua (achine is e0ecuted success,uy.
Ex No' (B
SH"+ THE IRT#%$ M%!HINE MIR%TI"N &%SED "N THE !ERT%IN !"NDITI"N FR"M "NE N"DE T" THE "THER.
D%TE'
"&-E!TIE'
5! earn virtua (achine (i'rati!n +ased !n the certain c!nditi!n ,r!( !ne n!de t! the !ther.
PR"!ED#RE'
5! de(!nstrate virtua (achine (i'rati!n8 t)! (achines (ust +e c!n,i'ured in !ne c!ud.
MIR%TI"N $IMIT%TI"NS
Openstac6 has t)! c!((ands speci,ic t! virtua (achine (i'rati!n@
n!va (i'rate _UUID
n!va ive(i'rati!n _UUID _COMPU5E>OS5
5he n!va (i'rate c!((and shuts d!)n an instance t! (!ve it t! an!ther hypervis!r.
5he instance is d!)n ,!r a peri!d !, ti(e and sees this as a re'uar shutd!)n.
It is n!t p!ssi+e t! speci,y the c!(pute h!st y!u )ant t! (i'rate the instance t!.
Read !n t! see h!) y!u can d! that the dirty )ay.
5his c!((and d!es n!t reuire shared st!ra'e8 the (i'rati!ns can ta6e a !n'ti(e.
5he Openstac6 custer ch!!ses the tar'et hypervis!r (achine +ased !n the ,ree res!urces and avaia+iity.
5he (i'rate c!((and )!r6s )ith any type !, instance.
5he M c!c6 has n! issues.
5he n!va ive(i'rati!n c!((and has a(!st n! instance d!)nti(e.
5he instance is suspended and d!es n!t see this as a shutd!)n.
5he ive(i'rati!n ets y!u speci,y the c!(pute h!st y!u )ant t! (i'rate t!8 h!)ever )ith s!(e i(itati!ns.
5his reuires shared st!ra'e8 instances )ith!ut a c!n,i'drive )hen +!c6 st!ra'e is used8 !r v!u(e+ac6ed instances.
5he (i'rati!n ,ais i, there are n!t en!u'h res!urces !n the tar'et hypervis!r
5he M c!c6 (i'ht +e !,,.
>ere are s!(e e0a(pes )hen t! use )hich !pti!n@
I, it is i(p!rtant t! ch!!se the c!(pute h!st !r t! have very itte d!)nti(e d!)nti(e y!u need t! use then!va ive(i'rati!n c!((and.
I, y!u d!n?t )ant t! ch!!se the c!(pute h!st8 !r y!u have a c!n,i'drive ena+ed8 y!u need t! use the n!va (i'rate c!((and.
I, y!u need t! speci,y the c!(pute h!st and y!u have a c!n,i'drive ena+ed8 y!u need t! (anuay (i'rate the (achine8 !r use a dirty tric6 t! ,!! n!va (i'rate.
Hyper)isor !apacity
Be,!re y!u d! a (i'rati!n8 chec6 i, the hypervis!r h!st has en!u'h ,ree capacity ,!r the M y!u )ant t! (i'rate@ n!va h!stdescri+e c!(pute/#
E0a(pe !utput@ 333333 >OS5 PRO1EC5 cpu (e(!ryQ(+ dis6Q'+ 333333 c!(pute/# t!ta 7" 4$-==# 4%-= c!(pute/# usedQn!) "" -$$$#" =%- c!(pute/# usedQ(a0 "" /$447= $/%- c!(pute/# "...F#-== $ 4$- -# c!(pute/# "...F#$%" -# "4#7 7-
In this ta+e8 the ,irst r!) sh!)s the t!ta a(!unt !, res!urces avaia+e !n the physica server. 5he sec!nd ine sh!)s the currenty used res!urces. 5he third ine sh!)s the (a0i(u( used res!urces. 5he ,!urth ine and +e!) sh!)s the res!urces avaia+e ,!r each pr!9ect. I, the M ,av!r ,its !n this hypervis!r8 c!ntinue !n )ith the (anua (i'rati!n. I, n!t8 ,ree up s!(e res!urces !r ch!!se an!ther c!(pute server. I, the hypervis!r n!de ac6s en!u'h capacity8 capacity8 the (i'rati!n )i ,ai. 1$i)e2 migration *ith no)a li)e8migration
5he ive(i'rati!n c!((and )!r6s )ith the ,!!)in' types !, v(?s2st!ra'e@
Shared st!ra'e@ B!th hypervis!rs have access t! shared st!ra'e.
B!c6 st!ra'e@ N! shared st!ra'e is reuired. Instances are +ac6ed +y i(a'e +ased
r!!t dis6s. Inc!(pati+e )ith read!ny devices such as CDROMs and C!n,i'urati!n Drive c!n,i'Qdrive.
!u(e st!ra'e@ N! shared st!ra'e is reuired. Instances are +ac6ed +y iSCSI v!u(es rather than ephe(era dis6. 5he ive(i'rati!n c!((and reuires the sa(e CPU !n +!th hypervis!rs. It is
p!ssi+e t! set a 'eneric CPU ,!r the M?s8 !r a 'eneric set !, CPU ,eatures. 5his h!)ever d!es n!t )!r6 !n versi!ns !)er than i! due t! a +u' )here N!va c!(pares the actua CPU instead !, the virtua CPU. In (y case8 a the hypervis!r (achines are the sa(e8 uc6y (e. 5his is ,i0ed in i! !r ater ater.. On versi! versi!ns ns !der !der than than i!8 i!8 the C!(pu C!(pute te servic servicee d!es d!es n!t use i+vir i+virt?s t?s ive ive (i'rati!n ,uncti!naity +y de,aut8 there,!re 'uests are suspended +e,!re (i'rati!n and (i'ht e0perience severa (inutes !, d!)nti(e. 5his is +ecause there is a ris6 that the (i'rati!n pr!cess )i never end. 5his can happen i, the 'uest !peratin' syste( uses +!c6s !n the dis6 ,aster than they can +e (i'rated. 5! ena+e true ive (i'rati!n usin' i+virt?s (i'rate ,uncti!naity8 ,uncti!naity8 see the Openstac6 d!cu(entati!n in6ed +e!). +e!). Share0 storage C olume ,ac>e0 instances
A ive(i'rati!n is very si(pe. Use the ,!!)in' c!((and )ith an instance UUID and the na(e !, the c!(pute h!st@ no)a li)e8migration ##ID !"MP#TE8H"ST
I, y!u have shared st!ra'e8 !r i, the instance is v!u(e +ac6ed8 this )i send the instances (e(!ry RAM c!ntent !ver t! the destinati!n h!st. 5he s!urce hypervis!r 6eeps trac6 !, )hich (e(!ry pa'es are (!di,ied !n the s!urce )hie the trans,er is in pr!'ress. Once the initia +u6 trans,er is c!(pete8 pa'es chan'ed in the (eanti(e are trans,erred a'ain a'ain.. 5h 5his is is d!n d!nee repeat repeated edy y )ith )ith idea idea y y ever ever s( s(a ae err incre incre(e (ent nts. s. As !n' !n' as the di,,erences can +e trans,erred ,aster than the s!urce M dirties (e(!ry pa'es8 at s!(e p!int the s!urce M 'ets suspended. ;ina di,,erences are sent t! the tar'et h!st and an identica (achine started there. At the sa(e ti(e the virtua net)!r6 in,rastructure ta6es care !, a tra,,ic +ein' directed t! the ne) virtua (achine. Once the repace(ent (achine is runnin'8 the suspended s!urce instance is deeted. Usuay the actua hand!ver ta6es pace s! uic6y and sea(essy that a +ut very ti(e sensitive appicati!ns ever n!tice anythin'. !u can chec6 this +y startin' a pin' t! the M y!u are ive(i'ratin'. It )i stay !nine and )hen the M is suspended and resu(ed !n the tar'et hypervis!r8 the pin' resp!nses )i ta6e a +it !n'er.
&loc> ,ase0 storage 188,loc>8migrate2
I, y!u d!n?t have shared st!ra'e and the M is n!t +ac6ed +y a v!u(e as r!!t dis6 i(a'e +ased M?s a ive(i'rati!n reuires an e0tra para(eter@ n!va ive(i'rati!n +!c6 (i'rate _UUID _COMPU5E>OS5
5he pr!cess is a(!st e0acty the sa(e as descri+ed a+!ve. 5here is !ne e0tra step h!)ev h!)ever er.. Be,! Be,!re re the (e(! (e(!ry ry c!nte c!ntents nts is sent sent the dis6 dis6 c!nten c!ntentt is c!pie c!pied d !ver !ver8 )ith! )ith!ut ut d!)nti(e. *hen the M is suspended8 +!th the (e(!ry c!ntents and the dis6 c!ntents di,,erence t! the earier c!py are sent !ver. 5he suspend acti!n ta6es !n'er and (i'ht +e n!tica+e as d!)nti(e. 5he +!c6(i'rate !pti!n is inc!(pati+e )ith read !ny devices
such as ISO CD2DD drives and the C!n,i' Drive.
Migration *ith no)a migrate
5he n!va (i'rate c!((and shuts d!)n an instance8 c!pies !ver the dis6 t! a hypervis!r )ith en!u'h ,ree res!urces8 starts it up there and re(!ves it ,r!( the s!urce hypervis!r. 5he M is shut d!)n and )i +e d!)n as !n' as the c!pyin'. *ith a (i'rate8 the Openstac6 custer ch!!ses an c!(puteservice ena+ed hypervis!r )ith the (!st res!urces avaia+e. 5his )!r6s )ith any type !, instance8 )ith any type !, +ac6end st!ra'e.
A (i'rate is even si(per than a ive(i'rati!n. >ere?s the synta0@ n!va (i'rate _UUID
5his is per,ect ,!r instances that are part !, a custered service8 !r )hen y!u have schedued and c!((unicated d!)nti(e ,!r that speci,ic M. 5he d!)nti(e is dependent !n the si
As seen a+!ve8 )e cann!t (i'rate virtua (achines t! a speci,ic c!(pute n!de i, the c!(pute n!de d!es n!t have shared st!ra'e and the virtua (achine has a c!n,i'drive ena+ed. !u can ,!rce the Openstac6 custer t! ch!!se a speci,ic hypervis!r +y disa+in' the n!vac!(pute service !n a the !ther hypervis!rs. 5he M?s )i 6eep runnin' !n there8 !ny ne) virtua (achines and (i'rati!ns are n!t p!ssi+e !n th!se hypervis!rs. I, y!u have a !t !, creatin' and re(!vin' !, (achines in y!ur Openstac6 Openstac6 C!ud8 this (i'ht +e a +ad idea. I, y!u use Anti A,,inity A,,inity Gr!ups8 v(?s created in there )i as! ,ai t! start dependin' !n the type !, A,,inity Gr!up. 5here,!re8 use this !pti!n )ith cauti!n. I, )e
have 4 c!(pute n!des8 c!(pute/# t! c!(pute/" and )e )ant t! (i'rate the (achine t! c!(pute/"8 )e need t! disa+e the n!vac!(pute service !n a !ther hypervis!rs.
;irst chec6 the state !, the custer@ n!va serviceist +inary n!vac!(pute ] !r n!vac!nduct!r8 n!vacert8 n!vac!ns!eauth8 n!vascheduer
E0a(pe !utput@ 333333333 Id Binary >!st b!ne Status State UpdatedQat Disa+ed Reas!n 333333333 : n!vac!(pute c!(pute/# OS$ ena+ed up -#$4#7$/5$:@#"@-:.###### = n!vac!(pute c!(pute/$ OS- ena+es up -#$4#7$/5$:@#-@"%.###### % n!vac!(pute c!(pute/- OS- ena+ed up -#$4#7$/5$:@#-@4#.###### N!ne $# n!vac!(pute c!(pute// OS- ena+ed up -#$4#7$/5$:@#-@4#.###### $$ n!vac!(pute c!(pute/" OS$ disa+ed up -#$4#7$/5$:@#-@"%.###### Mi'rati!ns Ony 333333333
In this e0a(pe )e have 4 c!(pute n!des8 !, )hich !ne is disa+ed )ith reas!n Mi'rati!ns Mi'rati!ns Ony. In !ur case8 +e,!re )e started (i'ratin' (i'ratin' )e have ena+ed n!va c!(pute !n that hypervis!r and disa+ed it !n a the !ther hypervis!rs@ n!va servicedisa+e c!(pute/# n!vac!(pute reas!n ?(i'rati!n t! speci,ic hypervis!r the dirty )ay? n!va servicedisa+e c!(pute/$ n!vac!(pute reas!n ?(i'rati!n t! speci,ic hypervis!r the dirty )ay? etc...
N!) e0ecute the n!va (i'rate c!((and. Since y!u?ve disa+ed a c!(pute hypervis!rs e0cept the tar'et hypervis!r8 that !ne )i +e used as (i'rati!n tar'et. A ne) virtu virtua a (ach (achine iness creat created ed durin durin' ' the the (i'ra (i'rati ti!n !n )i )i as! as! +e spa) spa)ned ned !n that that speci speci,i ,icc hypervis!r.*hen the (i'rati!n is ,inished8 ena+e a the !ther c!(pute n!des@ n!va serviceena+e c!(pute/# n!vac!(pute n!va serviceena+e c!(pute/$ n!vac!(pute etc...
In !ur case8 )e )!ud disa+e the c!(pute/" +ecause it is ,!r (i'rati!ns !ny. 5his is a +it dirty and (i'ht cause pr!+e(s i, y!u have (!nit!rin' !n the custer state !r spa)n a !t !, (achines a the ti(e.
Manual migration to a specific compute no0e
As seen a+!ve8 )e cann!t (i'rate virtua (achines t! a speci,ic c!(pute n!de i, the c!(pute n!de d!es n!t have shared st!ra'e and the virtua (achine has a c!n,i'drive ena+ed. Since Openstac6 is 9ust a +unch !, )rappers ar!und native Linu0 t!!s8 )e can (anuay (i'rate the (achine and update the N!va data+ase a,ter)ards. D! n!te that this part is speci,ic t! the st!ra'e y!u use. In this e0a(pe )e use !ca st!ra'e !r8 a !ca ,!der !n an N;S (!unt n!t shared )ith !ther c!(pute n!des and i(a'e +ac6ed instances. In (y case8 I needed t! (i'rate an i(a'e+ac6ed +!c6 st!ra'e st! ra'e instance insta nce t! a n!nshared st!ra'e n!de8 +ut the instance had a c!n,i'drive ena+ed. Disa+in' the c!(pute service every)here every)here is n!t an !pti!n8 since the custer )as 'ettin' a+!ut a hundred ne) M?s every 4 (inutes and that )!ud !ver!ad the hypervis!r n!de. 5his e0a(pe (anuay (i'rates a M ,r!( c!(pute/# t! c!(pute/". 5hese n!des are in the sa(e net)!r6 and can access !ne an!ther via SS> 6eys +ased !n their h!stna(e. Shut d!)n the M ,irst@ n!va st!p _MQUUID
As! detach any v!u(es@ n!va v!u(edetach _MQUUID _OLUMEQUUID
Use the n!va sh!) c!((and t! see the speci,ic hypervis!r the M is runnin' !n@ n!va sh!) UUID 'rep hypervis!r
E0a(pe !utput@ OSEH5SRA55R@hypervis!rQh!stna(e c!(pute/#
L!'in t! that hypervis!r via SS>. Navi'ate t! the ,!der )here this instance is !cated8 in !ur case8 case82var2i+2n!vac!(pute2instances2_UUID 2var2i+2n!vac!(pute2instances2_UUID.. 5he instance is +!!ted ,r!( an i(a'e +ased r!!t dis68 na(ed dis6. e(u in !ur case di,,s the r!!t dis6 ,r!( the i(a'e the M )as created ,r!(. 5here,!re the ne) hypervis!r as! needs that +ac6in' i(a'e. ;ind !ut )hich ,ie is the +ac6in' i(a'e@ cd 2var2i+2n!vac!(pute2instances2UUID 2var2i+2n!vac!(pute2instances2UUID22 e(ui(' in,! dis6 ] dis6 is the ,iena(e !, the instance r!!t dis6
E0a(pe !utput@ i(a'e@ dis6 ,ie ,!r(at@ c!)virtua si
c!(pat@ $.$ a
5he ,ie 2var2i+2n!vac!(pute2instances2Q+ase2d##",:,=d/,:%a#4/,ad4,%e4"a"aed%e-=7"47$ is the +ac6in' dis6. N!te that the !n' ,iena(e is n!t a UUID +ut a chec6su( !, the speci,ic i(a'e versi!n. In (y case it is a ra) dis6@ e(ui(' in,! 2var2i+2n!vac!(pute2instances2Q+a 2var2i+2n!vac!(pute2instances2Q+ase2d##...F7$ se2d##...F7$
E0a(pe !utput@ i(a'e@ 2var2i+2n!vac!(pute2instances2Q+a 2var2i+2n!vac!(pute2instances2Q+ase2d##...F se2d##...F 7$ ,ie ,!r(at@ ra) virtua si
Chec6 the tar'et hypervis!r ,!r the e0istence !, that i(a'e. I, it is n!t there8 c!py that ,ie t! the tar'et hypervis!r ,irst@ rsync r pr!'ress 2var2i+2n!vac!(pute2instances2Q+ase2d##...F7$ e ssh c!(pute/"@2var2i+2n!vac!(pute2instances2Q+ase2d##...F7$
On the tar'et hypervis!r8 set the c!rrect per(issi!ns@ ch!)n n!va@n!va 2var2i+2n!vac!(pute2instances2Q+ 2var2i+2n!vac!(pute2instances2Q+ase2d##...F7$ ase2d##...F7$
C!py the instance ,!der t! the ne) hypervis!r@ cd 2var2i+2n!vac!(pute2instances 2 var2i+2n!vac!(pute2instances22 rsync r pr!'ress _MQUUID e ssh c!(pute/"@2var2i+2n!vac!(pute2instances2 c!(pute/"@2var2i+2n!vac!(pute2instances2
Set the c!rrect per(issi!ns !n the ,!der !n the tar'et hypervis!r@ ch!)n n!va@n!va 2var2i+2n!vac!(pute2instances2_ 2var2i+2n!vac!(pute2instances2_MQUUID MQUUID ch!)n n!va@n!va 2var2i+2n!vac!(pute2instances2_ 2var2i+2n!vac!(pute2instances2_MQUUID2dis6.in,! MQUUID2dis6.in,! ch!)n n!va@n!va 2var2i+2n!vac!(pute2instances2i+virt.0( ch!)n i+virt@6v( 2var2i+2n!vac!(pute2instances2_M 2var2i+2n!vac!(pute2instances2_MQUUID2c!ns!e.!' QUUID2c!ns!e.!' ch!)n i+virt@6v( 2var2i+2n!vac!(pute2instances2_M 2var2i+2n!vac!(pute2instances2_MQUUID2dis6 QUUID2dis6 ch!)n i+virt@6v( 2var2i+2n!vac!(pute2instances2_M 2var2i+2n!vac!(pute2instances2_MQUUID2dis6.c!n,i' QUUID2dis6.c!n,i'
I, y!u use !ther userna(es and 'r!ups8 chan'e th!se in the c!((and.
L!' in t! y!ur data+ase server. In (y case that is a MySL Gaera custer. Start up a MySL c!((and pr!(pt in the n!vadata+ase (ys n!va
E0ecute the ,!!)in' c!((and t! update the n!va data+ase )ith the ne) hypervis!r ,!r this M@ update instances set n!deX?c!(pute/"?8 h!stXn!de )here uuidX?_MQUUID?
5his )as tested !n an Ice>!use data+ase sche(e8 !ther versi!ns (i'ht reuire !ther ueries.
Use the n!va sh!) c!((and t! see i, the ne) hypervis!r is set. I, s!8 start the M@ n!va start _MQUUID
Attach any v!u(es that )ere detached earier@ n!va v!u(eattach _MQUUID _OLUMEQUUID
Use the c!ns!e t! chec6 i, it a )!r6s@ n!va 'etvncc!ns!e _MQUUID n!vnc
D! n!te that y!u (ust chec6 the ,ree capacity y!urse,. 5he M )i )!r6 i, there is n!t en!u'h en!u'h capac capacit ity y8 +ut y!u d! run in t! )eir )eird d issue issuess )ith )ith the the hyp hyper ervi vis!r s!r i6e +ad +ad per,!r(ance !r 6ied pr!cesses OOM?s.
RES#$T'
5hus the virtua (achine (i'rati!n +ased !n the certain c!nditi!n ,r!( !ne n!de t! the !ther )as e0ecuted success,uy.
Ex No' ((
FIND PR"!ED#RE T" INST%$$ ST"R%E !"NTR"$$ER %ND INTER%!T +ITH IT
D%TE'
"&-E!TIE'
5! ,ind pr!cedure t! insta st!ra'e c!ntr!er and interact )ith it.
PR"!ED#RE'
"penStac> &loc> Storage
5he OpenStac6 B!c6 St!ra'e service cinder adds persistent st!ra'e t! a virtua (achine. (achine. B!c6 St!ra'e pr!vides an in,rastructure in,rastructure ,!r (ana'in' (ana'in' v!u(es8 and interacts )ith OpenStac6 OpenStac6 C!(pute t! pr!vide v!u(es ,!r instances. 5he service as! ena+es (ana'e(ent (ana'e(ent !, v!u(e snapsh!ts8 and v!u(e types. 5he B!c6 St!ra'e service c!nsists !, the ,!!)in' c!(p!nents@ cin0er8api
Accepts API reuests8 and r!utes the( t! the cinderv!u(e ,!r acti!n.
cin0er8)olume
Interacts directy )ith the B!c6 St!ra'e service8 and pr!cesses such as the cinder scheduer. It as! interacts )ith these pr!cesses thr!u'h a (essa'e ueue. 5he cinderv!u(e service resp!nds t! read and )rite reuests sent t! the B!c6 St!ra'e service t! (aintain state. It can interact )ith a variety !, st!ra'e pr!viders thr!u'h a driver architecture.
cin0er8sche0uler 0aemon
Seects the !pti(a st!ra'e pr!vider n!de !n )hich t! create the v!u(e. A si(iar c!(p!nent t! the n!vascheduer.
Messaging ueue
R!utes in,!r(ati!n +et)een the B!c6 St!ra'e pr!cesses.
Install an0 configure controller no0e
5his secti!n descri+es h!) t! insta and c!n,i'ure the B!c6 St!ra'e service8 c!de na(ed cinder8 !n the c!ntr!er n!de. 5his service reuires at east !ne additi!na st!ra'e n!de that pr!vides v!u(es t! instances.
To configure prereuisites
Be,!re y!u insta and c!n,i'ure the B!c6 St!ra'e service8 y!u (ust create a data+ase8 service credentias8 and API endp!ints. $. 5! create the data+ase8 c!(pete these steps@ a. Use the data+ase access cient t! c!nnect t! the data+ase server as the r!!t user@ _ (ys u r!!t p
-. Create the cinder data+ase@ CREA5E CREA 5E DA DA5ABASE 5ABASE cinder cin der a. Grant pr!per access t! the cinder data+ase@ GRAN5 ALL PRIILEGES ON cinder.Z 5O ?cinder?&?!cah!st? T IDEN5I;IED B ?C!D/12D3'ASS ?? GRAN5 ALL PRIILEGES ON cinder.Z 5O ?cinder?&?V? T IDEN5I;IED B ?C!D/12D3'ASS ??
Repace CINDERQDBPASS CINDERQDBPASS )ith a suita+e s uita+e pass)!rd. pass)! rd. +. E0it the data+ase access cient. /. S!urce the ad(in credentias t! 'ain access t! t ! ad(in!ny CLI c!((ands@ _ s!urce ad(in!penrc.sh
". 5! create the service credentias8 c!(pete these steps@ a. Create a cinder user@ _ 6eyst!ne usercreate na(e cinder pass C!D/12'ASS 333 Pr!perty aue 333 e(ai ena+ed 5rue id ==$a+-de",:%"$e:%4#"a:4%a=//#=+e na(e cinder userna(e cinder 333
Repace C!D/12'ASS )ith a suita+e pass)!rd. +. Add the ad(in r!e t! the cinder user@ _ 6eyst!ne userr!eadd user cinder tenant service r!e ad(in
c. Create the cinder service entities@ _ 6eyst!ne servicecreate na(e cinder type v!u(e T descripti!n YOpenStac6 B!c6 St !ra'eY
333 Pr!perty aue 333 descripti!n OpenStac6 B!c6 St!ra'e ena+ed 5rue id $e"%"c/e--a-"+aaa,ca,:::d"d"7: $e"%"c/e--a-"+aaa,ca,:::d"d"7:e+ e+ na(e cinder type v!u(e 333 _ 6eyst!ne servicecreate na(e cinderv- type v!u(ev- Tdescripti!n YOpenStac6 B!c6 St!ra'eY 333 Pr!perty aue 333 descripti!n OpenStac6 B!c6 St!ra'e ena+ed 5rue id $7e#/=e""%c%"+"#=7=-::,$d=#$ed+4 na(e cinderv- type v!u(ev- 333
". Create the B!c6 St!ra'e service API endp!ints@ _ 6eyst!ne endp!intcreate T serviceid _6eyst!ne serviceist a)6 ?2 v!u(e 2 [print _-\? T pu+icur http@22controller @=::72v$2VTtenantQidTs @=::72v$2VTtenantQidTs T internaur http@22controller @=::72v$2VTtenantQidTs @=::72v$2VTtenantQidTs T ad(inur http@22controller @=::72v$2VTtenantQidTs @=::72v$2VTtenantQidTs T re'i!n re'i!nOne 333 Pr!perty aue 333 ad(inur http@22c!ntr!er@=::72v$2VtenantQids id d$+:-%$a-d:%"e-7%7/+/--c:,-a44a" internaur http@22c!ntr!er@=::72v$2VtenantQids pu+icur http@22c!ntr!er@=::72v$2VtenantQids re'i!n re'i!nOne serviceQid $e"%"c/e--a-"+aaa,ca,:::d"d"7:e+ 333 _ 6eyst!ne endp!intcreate T serviceid _6eyst!ne serviceist a)6 ?2 v!u(ev- 2 [print _-\? T pu+icur http@22controller @=::72v-2VTtenantQidTs @=::72v-2VTtenantQidTs T internaur http@22controller @=::72v-2VTtenantQidTs @=::72v-2VTtenantQidTs T ad(inur http@22controller @=::72v-2VTtenantQidTs @=::72v-2VTtenantQidTs T re'i!n re'i!nOne 333 Pr!perty aue 333 ad(inur http@22c!ntr!er@=::72v-2VtenantQids id #%:+"a7,c=+a""+"+$#d"=--d-d%e#:7 internaur http@22c!ntr!er@=::72v-2VtenantQids pu+icur http@22c!ntr!er@=::72v-2VtenantQids re'i!n re'i!nOne serviceQid $7e#/=e""%c%"+"#=7=-::,$d=#$ed+4 333
To install an0 configure &loc> Storage controller components $. Insta the pac6a'es@ ] apt'et insta cinderapi cinderscheduer pyth!ncindercient
-. Edit the 2etc2cinder2cinder.c!n, ,ie and c!(pete the ,!!)in' acti!ns@ a. In the data+aseF secti!n8 c!n,i'ure data+ase access@ data+aseF ... c!nnecti!n X (ys@22cinder@CINDERQDBP (ys@22cinder@CINDERQDBPASS&c!ntr!er2cinder ASS&c!ntr!er2cinder
Repace C!D/12D3'ASS )ith the pass)!rd y!u ch!se ,!r the B!c6 St!ra'e data+ase. +. In the DE;AUL5F secti!n8 c!n,i'ure Ra++itM (essa'e +r!6er access@ DE;AUL5F ... authQstrate'y X 6eyst!ne 6eyst!neQautht!6enF ... authQuri X http@22c!ntr!er@4###2v-.# identityQuri X http@22c!ntr!er@/4/4: ad(inQtenantQna(e X service ad(inQuser X cinder ad(inQpass)!rd X CINDERQPASS CINDERQPASS /. Repace CINDERQPASS CINDERQPASS )ith the pass)!rd y!u ch!se ,!r the t he cinder user in the Identity service. ". d. In the DE;AUL5F DE;AUL5F secti!n8 c!n,i'ure the (yQip !pti!n t! use the (ana'e(ent inter,ace IP address !, the c!ntr!er n!de@ DE;AUL5F ... (yQip X $#.#.#.$$ DE;AUL5F secti!n@ e. Opti!na 5! assist )ith tr!u+esh!!tin'8 ena+e ver+!se !''in' in the DE;AUL5F DE;AUL5F ... ver+!se X 5rue /. P!puate the B!c6 St!ra'e data+ase@ ] su s 2+in2sh c Ycinder(ana'e d+ syncY cinder
To finalie installation
$. Restart the B!c6 St!ra'e services@ ] service cinderscheduer restart ] service cinderapi restart -. By de,aut8 the U+untu pac6a'es create an SLite data+ase. Because this c!n,i'urati!n uses a SL data+ase server8 y!u can re(!ve the SLite data+ase ,ie@ ] r( , 2var2i+2cinder2cinder.site
Install an0 configure a storage no0e
5his secti!n descri+es h!) t! insta and c!n,i'ure st!ra'e n!des ,!r the B!c6 St!ra'e service. ;!r si(picity8 this c!n,i'urati!n re,erences !ne st!ra'e n!de )ith an e(pty !ca !ca +!c6 +!c6 st!ra' st!ra'ee devic devicee 2dev2 2dev2sd+ sd+ that that c!nta c!ntain inss a suita suita+ +ee parti partiti ti!n !n ta+e ta+e )ith )ith !ne partiti!n 2dev2sd+$ !ccupyin' the entire device. 5he service pr!visi!ns !'ica v!u(es !n this device usin' the LM driver and pr!vides the( t! instances via iSCSI transp!rt. !u can ,!!) these instructi!ns )ith (in!r (!di,icati!ns t! h!ri
!u (ust c!n,i'ure the st!ra'e n!de +e,!re y!u insta and c!n,i'ure the v!u(e service !n it. Si(iar t! the c!ntr!er n!de8 the st!ra'e n!de c!ntains !ne net)!r6 inter,ace !n the (ana'e(ent net)!r6. 5he st!ra'e n!de as! needs an e(pty +!c6 st!ra'e device !, suita+e si
/. C!py the c!ntents !, the 2etc2h!sts ,ie ,r!( the c!ntr!er n!de t! the st!ra'e n!de and add the ,!!)in' t! it@ ] +!c6$ $#.#.#."$ +!c6$
As! add this c!ntent t! the 2etc2h!sts ,ie !n a !ther n!des in y!ur envir!n(ent. ". Insta and c!n,i'ure N5P usin' the instructi!ns in the secti!n caed JOther n!desK. 4. Insta the LM pac6a'es@ ] apt'et insta v(-
Create the LM physica v!u(e 2dev2sd+$@ ] pvcreate 2dev2sd+$ Physica v!u(e Y2dev2sd+$Y success,uy created
:. Create the LM v!u(e 'r!up cinderv!u(es@ ] v'create cinderv!u(es 2dev2sd+$ !u(e 'r!up Ycinderv!u(esY success,uy created
5he B!c6 St!ra'e service creates !'ica v!u(es in this v!u(e 'r!up. Ony instances can access B!c6 St!ra'e v!u(es. >!)ever8 the underyin' !peratin' syste( (ana'es the devices ass!ciated )ith the v!u(es. By de,aut8 the LM v!u(e
scannin' t!! scans the 2dev direct!ry ,!r +!c6 st!ra'e devices that c!ntain v!u(es. I, tenants use LM !n their v!u(es8 the scannin' t!! detects these v!u(es and atte(pts t! cache the( )hich can cause a variety !, pr!+e(s )ith +!th the underyin' !peratin' syste( and tenant v!u(es. !u (ust rec!n,i'ure LM t! scan !ny the devices that c!ntain the cinderv!u(e v!u(e 'r!up. Edit the 2etc2v(2v(.c!n, ,ie and c!(pete the ,!!)in' acti!ns@ a. In the devices secti!n8 add a ,iter that accepts the 2dev2sd+ device and re9ects a !ther devices@ devices [ ... ,iter X Ya2sd+2Y8 Yr2.Z2YF
Each ite( in the ,iter array +e'ins )ith a ,!r accept !r r ,!r reect and incudes a re'uar e0pressi!n ,!r the device na(e. 5he array (ust end )ith r2.Z2 t! re9ect any re(ainin' devices. !u can use u se the )gs 8)))) c!((and t! test ,iters.
Install an0 configure &loc> Storage )olume components $. Insta the pac6a'es@ ] apt'et insta cinderv!u(e pyth!n(ysd+
-. Edit the 2etc2cinder2cinder.c!n, ,ie and c!(pete the ,!!)in' acti!ns@ /. In the data+aseF secti!n8 c!n,i'ure data+ase access@ data+aseF ... c!nnecti!n X (ys@22cinder@CINDERQDBP (ys@22cinder@CINDERQDBPASS&c!ntr!er2cinder ASS&c!ntr!er2cinder
Repace C!D/12D3'ASS )ith the pass)!rd y!u ch!se ,!r the B!c6 St!ra'e data+ase. a. In the DE;AUL5F secti!n8 c!n,i'ure Ra++itM (essa'e +r!6er access@ DE;AUL5F ... rpcQ+ac6end X ra++it ra++itQh!st X c!ntr!er ra++itQpass)!rd X RABBI5QP RABBI5QPASS ASS
Repace 1A33!#2'ASS 1A33!#2'ASS )ith the pass)!rd y!u ch!se ,!r the 'uest acc!unt in Ra++itM. c. In the DE;AUL5F and 6eyst!neQautht!6enF secti!ns8 c!n,i'ure Identity service access@ DE;AUL5F ... authQstrate'y X 6eyst!ne 6eyst!neQautht!6enF
... authQuri X http@22c!ntr!er@4###2v-.# identityQuri X http@22c!ntr!er@/4/4: ad(inQtenantQna(e X service ad(inQuser X cinder ad(inQpass)!rd X CINDERQP CINDERQPASS ASS
Repace C!D/12'ASS )ith the pass)!rd y!u ch!se ,!r the cinder user in the Identity service. d. In the DE;AUL5F secti!n8 c!n,i'ure the (yQip !pti!n@ DE;AUL5F ... (yQip X MANAGEMEN5QIN5ER; MANAGEMEN5QIN5ER;ACEQIPQADDRESS ACEQIPQADDRESS
Repace MAA"/M/#2!#/1F MAA"/M/#2!#/1FAC/2!'2ADD1/ AC/2!'2ADD1/SS SS )ith the IP address !, the (ana'e(ent net)!r6 inter,ace !n y!ur st!ra'e n!de8 typicay $#.#.#."$ ,!r the ,irst n!de in the e0a(pe architecture. e. In the DE;AUL5F secti!n8 c!n,i'ure the !cati!n !, the I(a'e Service@ DE;AUL5F ... 'anceQh!st X c!ntr!er
,. Opti!na 5! assist )ith tr!u+esh!!tin'8 ena+e ver+!se !''in' in the DE;AUL5F secti!n@ DE;AUL5F ... ver+!se X 5rue
To finalie installation
$. Restart the B!c6 St!ra'e v!u(e service incudin' its dependencies@ ] service t't restart ] service cinderv!u(e restart
-. By de,aut8 the U+untu pac6a'es create an SLite data+ase. Because this c!n,i'urati!n uses a SL data+ase server8 re(!ve the SLite data+ase ,ie@ ] r( , 2var2i+2cinder2cinder.site
erify operation $.5his secti!n descri+es h!) t! veri,y !perati!n !, the B!c6 St!ra'e service +y creatin' a v!u(e
S!urce the ad(in credentias t! 'ain access t! ad(in!ny CLI c!((ands@ _ s!urce ad(in!penrc.sh
-. List service c!(p!nents t! veri,y success,u aunch !, each pr!cess@ _ cinder serviceist 33333333 Binary >!st b!ne Status State UpdatedQat Disa+ed Reas!n 33333333 cinderscheduer c!ntr!er n!va ena+ed up -#$"$#$=5#$@/#@4".###### N!ne cinderv!u(e +!c6$ n!va ena+ed up -#$"$#$=5#$@/#@4:.###### N!ne 33333333
/. S!urce the de(! tenant credentias t! per,!r( the ,!!)in' steps as a n!n ad(inistrativetenant@ _ s!urce de(!!penrc.sh
". Create a $ GB v!u(e@ _ cinder create dispayna(e de(!v!u(e$ $ 333 Pr!perty aue 333 attach(ents F avaia+iityQ
4. eri,y eri,y creati!n and avaia+iity !, the v!u(e@ _ cinder ist 3333333 ID Status Dispay Na(e Si
!ur OpenStac6 envir!n(ent n!) incudes B!c6 St!ra'e.
RES#$T'
5hus the pr!cedure t! insta st!ra'e c!ntr!er and interact )ith !penstac6 service is e0ecuted success,uy. success,uy.
Ex No' (9
FIND PR"!ED#RE T" SET #P THE "NE N"DE H%D""P !$#STER
D%TE'
"&-E!TIE'
Set up the !ne n!de >ad!!p custer.
PR"!ED#RE'
$ Instain' 1ava >ad!! >ad!!p p is a ,ra(e ,ra(e)! )!r6 r6 )rit )ritte ten n in 1ava 1ava ,!r runnin runnin' ' appi appica cati ti!ns !ns !n ar' ar'ee custe custers rs !, c!((!dity hard)are. >ad!!p needs 1ava 7 !r a+!ve t! )!r6. Step $@ D!)n!ad tar and e0tract D!)n!ad 1d6 tar.'< ,ie ,!r inu07- +it8 e0tract it int! J2usr2!caK ] cd 2!pt ] sud! tar 0vp<, 2h!(e2itad(in2D!)n!ads29d6=u4inu007".tar 2h!(e2itad(in2D!)n!ads29d6=u4inu007".tar.'< .'< ] cd 2!pt29d6$.=.#Q#4
Step -@ Set Envir!n(ents
Open the J2etc2pr!,ieK ,ie and Add the ,!!)in' ine as per the versi!n
Set a envir!n(ent ,!r 1ava
Use the r!!t user t! save the 2etc2pr!,ie !r use 'edit instead !, vi .
5he ?pr!,ie? ,ie c!ntains c!((ands that !u'ht t! +e run ,!r !'in shes
] sud! vi 2etc2pr!,ie ]insert ]in sert 1A 1A AQ>OME 1AAQ>OMEX2!pt29d6$.=.#Q#4 ]in PA5> varia+e 9ust append at the end !, the ine PA5>X_PA5>@_1AAQ>OME2+in ]Append 1A 1A AQ>OME at end en d !, the e0p!rt e0p!r t state(ent e0p!rt PA5> 1AAQ>OME save the ,ie usin' +y pressin' JEscK 6ey ,!!)ed +y @)
Step /@ S!urce the 2etc2pr!,ie ] s!urce 2etc2pr!,ie
Step "@ Update the 9ava aternatives $. By de,aut OS )i have a !pen 9d6. Chec6 +y J9ava versi!nK. !u )i +e pr!(pt J!pen1DK -. I, y!u as! have !pen9d6 instaed then y!u? need t! update the 9ava aternatives@ /. I, y!ur syste( has (!re than !ne versi!n !, 1ava8 c!n,i'ure )hich !ne y!ur syste( causes +y enterin' the ,!!)in' c!((and in a ter(ina )ind!) ". By de,aut OS )i have a !pen 9d6. Chec6 +y J9ava versi!nK. !u )i +e pr!(pt J1ava>!tSp!t5M 7"Bit ServerK ] updateaternatives insta Y2usr2+in29avaY 9ava Y2!pt29d6$.=.#Q#42+in29avaY $ ] updateaternatives c!n,i' 9ava type seecti!n nu(+er@ ] 9ava versi!n
- c!n,i'ure ssh
>ad!!p reuires SS> access t! (ana'e its n!des8 i.e. re(!te (achines pus y!ur !ca (achine i, y!u )ant t! use >ad!!p !n it )hich is )hat )e )ant t! d! in this e0ercise. ;!r !ur sin'en!de setup !, >ad!!p8 )e there,!re need t! c!n,i'ure SS> access t! !cah!st 5he need t! create a Pass)!rdess SS> ey 'enerati!n +ased authenticati!n is s! that the (aster n!de can then !'in t! save n!des and the sec!ndary n!de t! start2st!p the( easiy )ith!ut any deays ,!r authenticati!n
I, y!u s6ip this step8 then have t! pr!vide pass)!rd
Generate Generate an SS> 6ey ,!r the user. user. 5hen Ena+e pass)!rdess SS> access t! y!sud! apt'et insta !pensshserver !u )i +e as6ed t! enter pass)!rd8 r!!t&a+c F] ssh !cah!st r!!t&a+cF] ssh6ey'en r!!t&a+cF] sshc!pyid i !cah!st A,ter a+!ve - steps8 !u !u )i +e c!nnected )ith!ut )it h!ut pass)!rd8 r!!t&a+cF] ssh !cah!st r!!t&a+cF] e0it
=2 Ha0oop installation
N!) D!)n!ad D!)n!a d >ad!!p ,r!( the !,,icia Apache8 pre,era+y a sta+e reease versi!n !, >ad!!p -.:.0 and e0tract the c!ntents !, the >ad!!p pac6a'e t! a !cati!n !, y!ur ch!ice. ;!r e0a(pe8 ch!!se !cati!n as J2!pt2K
Step (' D!)n!ad the tar.'< ,ie !, atest versi!n >ad!!p had!!p-.:.0 ,r!( the !,,icia
site . Step 9@ E0tract untar the d!)n!aded ,ie ,r!( this c!((ands t! 2!pt2+i'data
r!!t&a+cF] cd 2!pt r!!t&a+c2!ptF] sud! tar 0vp<, 2h!(e2itad(in2D!)n!ads2had!!p-.:.#.tar.'< r!!t&a+c2!ptF] cd had!!p-.:.#2 Li6e 9ava8 update >ad!p envir!n(ent varia+e in 2etc2pr!,ie ] sud! vi 2etc2pr!,ie ]insert >ADOOPQPRE;IH >ADOOPQPRE;IHX2!pt2had!!p-.:.# ]in PA5> varia+e 9ust append at the end !, the ine PA5>X_PA5>@_>ADOOPQPRE;IH2+in ]Append >ADOOPQPRE;IH at end !, the e0p!rt state(ent e0p!rt PA5> 1AAQ>OME >ADOOPQPRE;IH save the ,ie usin' +y pressin' JEscK 6ey ,!!)ed +y @) Step =' Source the CetcCprofile
] s!urce 2etc2pr!,ie eri,y >ad!!p >ad! !p instaati!n instaat i!n ] cd _>ADOOPQPRE;IH ] +in2had!!p versi!n =.(2 Mo0ify the Ha0oop !onfiguration Files
In this this sect secti! i!n8 n8 )e )i )i c!n, c!n,i' i'ur uree the the dire direct ct!r !ry y )her )heree >ad! >ad!!p !p )i )i st!r st!ree its its c!n,i'urati!n ,ies8 the net)!r6 p!rts it istens t!8 etc. Our setup )i use >ad!!p Dist Distri+ ri+ute uted d ;ie ;ie Syste Syste(8 (8>D >D;S ;S8 8 even even th!u'h th!u'h )e are usin' usin' !ny !ny a sin'e sin'e !ca !ca (achine.
Add the ,!!)i ,!!)in' n' pr!perti pr!perties es in the vari!us had!!p c!n,i'urat c!n,i'urati!n i!n ,ies ,ies )hich )hich is avaia+e under _>ADOOPQPRE;IH2etc2had!!p2
c!resite.0(8 hd,ssite.0(8 (apredsite.0( ^ yarnsite.0(
#p0ate -a)a ha0oop path to the Ha0oop en)ironment en)ironment file
] cd _>ADOOPQPRE;IH2etc2had!!p ] vi had!!penv.sh
Paste follo*ing line at ,eginning of the file
e0p!rt 1A 1A AQ>OMEX2usr2!ca29d6$.=.#Q#4 e0p!rt >ADOOPQPRE;IHX2!pt2had!!p-.:.#
Mo0ify the core8site.xml core8site.xml
] cd _>ADOOPQPRE;IH2etc2had!!p ] vi c!resite.0(
Paste follo*ing ,et*een VconfigurationL tags
Wc!n,i'urati!n Wpr!perty Wna(e,s.de,aut;SW2na(e Wvauehd,s@22!cah!st@%###W2vaue W2pr!perty W 2c!n,i'urati!n
Mo0ify the h0fs8site.xml
] vi hd,ssite.0(
Paste follo*ing ,et*een VconfigurationL tags
Wc!n,i'urati!n Wpr!perty Wna(ed,s.repicati!nW2na(e Wvaue$W2vaue W2pr!perty W2c!n,i'urati!n
Q%RN configuration 8 Single No0e mo0ify the mapre08site.xml mapre08site.xml
] cp (apredsite.0(.te(pate (apredsite.0( ] vi (apredsite.0(
Paste follo*ing ,et*een VconfigurationL tags
Wc!n,i'urati!n Wpr!perty Wna(e(apreduce.,ra(e)!r6.na(eW2na(e WvaueyarnW2vaue W2pr!perty W2c!n,i'urati!n
Mo0iy yarn8site.xml
] vi yarnsite.0(
Paste follo*ing ,et*een VconfigurationL tags
Wc!n,i'urati!n Wpr!perty Wna(eyarn.n!de(ana'er.au0servicesW2na(e Wvaue(apreduceQshu,,eW2vaue W2pr!perty W2c!n,i'urati!n
Formatting the HDFS file8system )ia the NameNo0e
5he ,irst step t! startin' up y!ur >ad!!p instaati!n is ,!r(attin' the >ad!!p ,ies syste( )hich is i(pe(ented !n t!p !, the !ca ,ie syste( !, !ur JcusterK )hich incudes !ny !ur !ca (achine. *e need t! d! this the ,irst ti(e y!u set up a >ad!!p custer.
D! n!t ,!r(at a runnin' >ad!!p ,ie syste( as y!u )i !se a the data currenty in the custer in >D;S
r!!t&a+cF] cd _>ADOOPQPRE;IH r!!t&a+cF] +in2had!!p na(en!de ,!r(at
Start NameNo0e 0aemon an0 DataNo0e 0aemon' 1port BBJB2
r!!t&a+cF] s+in2startd,s.sh 5! 6n!) the runnin' dae(!ns 9ut type 9ps !r 2usr2!ca29d6$.=.#Q#42+in29ps
Start ResourceManager 0aemon an0 No0eManager 0aemon' 1port ;B;;2
r!!t&a+cF] s+in2startyarn.sh
To stop the running process
r!!t&a+cF] s+in2st!pd,s.sh 5! 6n!) the runnin' dae(!ns 9ut type 9ps !r 2usr2!ca29d6$.=.#Q#42+in29ps
Start ResourceManager 0aemon an0 No0eManager 0aemon' 1port ;B;;2
r!!t&a+cF] s+in2st!pyarn.sh
Ma>e the HDFS 0irectories reuire0 reuire0 to execute MapRe0uce 7o,s'
_ +in2hd,s d,s (6dir 2user _ +in2hd,s d,s (6dir 2user2(it
C!py the input ,ies int! the distri+uted ,iesyste(@
_ +in2hd,s d,s put Winputpath2Z 2input
Run s!(e !, the e0a(pes pr!vided@
_ +in2had!!p 9ar share2had!!p2(apreduce2had!! s hare2had!!p2(apreduce2had!!p(apreducee0a(pes-.4.$.9ar p(apreducee0a(pes-.4.$.9ar 'rep 2input 2!utput ?CSE?
E0a(ine the !utput ,ies@
C!py the !utput ,ies ,r!( the distri+uted ,iesyste( t! the !ca ,iesyste( and e0a(ine the(@ _ +in2hd,s d,s 'et !utput !utput _ cat !utput2Z !r
ie) the !utput ,ies !n the distri+uted ,iesyste(@
_ +in2hd,s d,s cat 2!utput2Z
RES#$T'
5hus settin' up !, !ne n!de >ad!!p custer is success,uy e0ecuted.
Ex No' (= M"#NT THE "NE N"DE H%D""P !$#STER #SIN F#SE D%TE'
"&-E!TIE'
5! (!unt the !ne n!de >ad!!p custer usin' ;USE
PR"!ED#RE'
>ad!! >ad!!p p Distr Distri+u i+ute ted d ;ie ;ie Syste Syste( ( >D; >D;S S is a distr distri+ i+ute uted8 d8 scaa scaa+ +ee ,ie ,ie syste syste( ( deve!ped as the +ac6end st!ra'e ,!r dataintensive >ad!!p appicati!ns. As such8 >D;S is desi'ned t! hande very ar'e ,ies )ith Y)rite!nceread(anyY access (!de. As >D;S is n!t a ,u ,u, ,ed ed'ed 'ed POSI POSIH H c!(pi c!(piant ant ,ie ,ie syste syste(8 (8 it cann!t cann!t +e direct directy y (! (!unt unted ed +y the !peratin' syste(8 and ,ie access )ith >D;S is d!ne via >D;S she c!((ands.
>!)ever8 >!)ever8 !ne can evera'e ;USE t! )rite a userand appicati!n that e0p!ses >D;S via a traditi!na ,ie syste( inter,ace. ,used,s is !ne such ;USE+ased appicati!n )hich a!)s y!u t! (!unt >D;S as i, it )ere a traditi!na Linu0 ,ie syste(. I, y!u )!ud i6e t! (!unt >D;S !n Linu08 y!u can insta ,used,s8 a!n' )ith ;USE as ,!!)s. N!)8 N!)8 insta ,used,s and a necessary dependencies dependen cies as ,!!)s.
To install fuse80fs on #,untu (9.B5 an0 higher'
_ )'et [email protected]!udera.c!(2!necic6insta2(averic62cdh/rep!sit!ryQ$.#Qa.de+
_ sud! dp6' i cdh/rep!sit!ryQ$.#Qa.de+
_ sud! apt'et update
_ sud! apt'et insta had!!p#.-#,use
"nce fuse80fs is installe0 go ahea0 an0 mount HDFS using F#SE as follo*s.
_ sud! had!!p,used,s d,s@22Wna(eQn!deQh!stna(e@Wna(en!deQp!rt W(!untQp!int
Once >D;S has +een (!unted at W(!untQp!int8 W(!untQp!int8 y!u can use (!st !, the traditi!na traditi!na ,iesyste( !perati!ns e.'.8 cp8 r(8 cat8 (v8 (6dir8 r(dir8 (!re8 scp. >!)ever8 rand!( )rite !perati!ns such as rsync8 and per(issi!n reated !perati!ns such as ch(!d8 ch!)n are n!t supp!rted in ;USE(!unted >D;S.
RES#$T'
5hus (!untin' the !ne n!de >ad!!p custer usin' ;USE is success,uy e0ecuted.
Ex No' (5
+RITE % PR"R%M T" #SE THE %PIS "F H%D""P T" INTER%!T +ITH IT
D%TE'
"&-E!TIE'
Pr!'ra( t! use >ad!!ps ;ie syste( API?
PR"!ED#RE'
Readin' data ,r!( and )ritin' data t! >ad!!p Distri+uted ;ie Syste( >D;S can +e d!ne in a !t !, )ays. N!) et us start s tart +y usin' the ;ieSyste( API t! create and )rite t! a ,ie in >D;S8 ,!!)ed +y an appicati!n t! read a ,ie ,r!( >D;S and )rite it +ac6 t! the !ca ,ie syste(.
Step (' "nce you ha)e 0o*nloa0e0 a test 0ataset *e can *rite an application to rea0 a file from the local file system an0 *rite the contents to Ha0oop Distri,ute0 File System.
pac6a'e c!(.had!!p.hd,s.)riter c!(.had!!p.hd ,s.)riter i(p!rt !r'.apache.had!!p.c!n,.C!n,i'ured i(p!rt !r'.apache.had!!p.uti.5!! i(p!rt 9ava.i!.Bu,,eredInputStrea( i(p!rt 9ava.i!.;ieInputStrea( i(p!rt 9ava.i!.InputStrea( i(p!rt 9ava.i!.OutputStrea( i(p!rt !r'.apache.had!!p.c!n,.C!n,i'urati!n i(p!rt !r'.apache.had!!p.,s.;ieSyste( i(p!rt !r'.apache.had!!p.,s.Path i(p!rt !r'.apache.had!!p.i!.IOUtis i(p!rt !r'.apache.had!!p.uti.5!!Runner pu+ic cass >d,s*riter e0tends C!n,i'ured C!n,i' ured i(pe(ents 5!! 5!! [ pu+ic static ,ina Strin' ;SQPARAMQNAME ;SQPARAMQNAME X Y,s.de,aut;SY pu+ic int runStrin'F runStrin' F ar's thr!)s E0cepti!n [ i, ar's.en'th W - [ Syste(.err.printnY>d,s*riter !ca input pathF hd,s !utput pathFY return $ \ Strin' !caInputPath X ar's#F Path !utputPath X ne) Pathar's$F C!n,i'urati!n c!n, X 'etC!n, Syste(.!ut.printnYc!n,i'ured ,iesyste( X Y 3 c!n,.'et;SQPARAMQNAME ;ieSyste( ,s X ;ieSyste(.'etc!n, i, ,s.e0ists!utputPath [
Syste(.err.printnY!utput path e0istsY return $ \ OutputStrea( !s X ,s.create!utputPath InputStrea( is X ne) Bu,,eredInputStrea(ne) ;ieInputStrea(!caInputPath IOUtis.c!pyBytesis8 !s8 c!n, return # \ pu+ic static v!id (ain Strin'F ar's thr!)s E0cepti!n [ int returnC!de X 5!!Runner.runne) 5!!Runner.runne) >d,s*riter8 ar's Syste(.e0itreturnC!de \ \ Step 9' Export the -ar file an0 run the co0e from terminal to *rite a sample file to HDFS.
trainin'&!cah!st F_ had!!p 9ar >d,s*riter.9ar c!(.had!!p.hd,s.)riter.>d,s*riter sa(pe.t0t 2user2trainin'2>d,s*riterQsa(pe.t0t
Step =' erify erify *hether the file is *ritten into HDFS an0 chec> the contents of the file.
trainin'&!cah!st F_ had!!p ,s cat 2user2trainin'2>d,s*riterQsa(pe.t0t
Step 5' Next *e *rite an application to rea0 the file *e 7ust create0 in Ha0oop Distri,ute0 File System an0 *rite its contents ,ac> to the local file system.
pac6a'e c!(.had!!p.hd,s.reader c!(.had!!p.hd ,s.reader i(p!rt 9ava.i!.Bu,,eredOutputStrea( i(p!rt 9ava.i!.;ieOutputStrea( i(p!rt 9ava.i!.InputStrea( i(p!rt 9ava.i!.OutputStrea( i(p!rt !r'.apache.had!!p.c!n,.C!n,i'urati!n i(p!rt !r'.apache.had!!p.c!n,.C!n,i'ured i(p!rt !r'.apache.had!!p.,s.;ieSyste( i(p!rt !r'.apache.had!!p.,s.Path i(p!rt !r'.apache.had!!p.i!.IOUtis i(p!rt !r'.apache.had!!p.uti.5!! i(p!rt !r'.apache.had!!p.uti.5!!Runner pu+ic cass >d,sReader e0tends e0 tends C!n,i'ured i(pe(ents i(pe(en ts 5!! 5!! [ pu+ic static ,ina Strin' ;SQPARAMQNAME ;SQPARAMQNAME X Y,s.de,aut;SY pu+ic int runStrin' F ar's a r's thr!)s E0cepti!n [ i, ar's.en'th W - [ Syste(.err.printnY>d,sReader hd,s input pathF !ca !utput pathFY return $ \ Path inputPath X ne) Pathar's#F
Strin' !caOutputPath X ar's$F C!n,i'urati!n c!n, X 'etC!n, Syste(.!ut.printnYc!n,i'ured ,iesyste( X Y 3 c!n,.'et;SQPARAMQNAME ;ieSyste( ,s X ;ieSyste(.'etc!n, InputStrea( is X ,s.!peninputPath OutputStrea( !s X ne) Bu,,eredOutputStrea(ne) ;ieOutputStrea(!caOutputPath IOUtis.c!pyBytesis8 !s8 c!n, return # \ pu+ic static v!id (ain Strin'F ar's thr!)s E0cepti!n [ int returnC!de X 5!!Runner.runne) >d,sReader8 ar's Syste(.e0itreturnC!de \ \ Step ' Export the -ar file an0 run the co0e from terminal to *rite a sample file to HDFS.
trainin'&!cah!st F_ had!!p 9ar >d,sReader.9ar c!(.had!!p.hd,s.reader.>d,sReader 2user2trainin'2>d,s*riterQsa(pe.t0t 2h!(e2trainin'2>d,sReaderQsa(pe.t0t
Step ' erify erify *hether the file is *ritten ,ac> into local file system.
trainin'&!cah!st F_ had!!p ,s cat 2user2trainin'2>d,s*riterQsa(pe.t0t ;ieSyste( ;ieSyste( is an a+stract cass that represents a 'eneric ,ie syste(. M!st >ad!!p ,ie syste( i(pe(entati!ns can +e accessed and updated thr!u'h the ;ieSyste( !+9ect. 5! create an instance !, the >D;S8 y!u ca the (eth!d ;ieSyste(.'et. 5he ;ieSyste(.'et (eth (eth!d !d )i )i !!6 !!6 at the the URI URI assi assi'n 'ned ed t! the the ,s.d ,s.de, e,au aut t;S ;S para para(e (ete terr !, the the >ad! >ad!!p !p c!n,i c!n,i'ur 'urat ati! i!n n ,ie ,iess !n y!u y!urr cass casspa path th and and ch!!se ch!!se the c!rrec c!rrectt i(pe i(pe(e (enta ntati! ti!n n !, the the ;ieSyste( cass t! instantiate. 5he,s.de,aut;S para(eter !, >D;S has the vaue hd,s@22. Once an instance !, the ;ieSyste( ;ieSyste( cass has +een created8 the >d,s*riter >d,s*riter cass cas the create (eth!d t! create a ,ie in >D;S. 5he create (eth!d return an OutputStrea( !+9ect8 )hich can +e (anipuated usin' n!r(a 1ava I2O (eth!ds. Si(iary >d,sReader cas the (eth!d !pen t! !pen a ,ie in >D;S8 )hich returns an InputStrea( !+9ect that can +e used t! read the c!ntents !, the ,ie. 5he ;ieSyste( API is e0tensive. 5! de(!nstrate s!(e !, the !ther (eth!ds avaia+e in the API8 )e can add s!(e err!r chec6in' t! the >d,s*riter and >d,sReader casses )e created.
To chec> *hether the file exists ,efore *e call create12 use'
+!!ean e0ists X ,s.e0istsinputPath ,s.e0 istsinputPath
To chec> *hether the path is a file use'
+!!ean is;ie X ,s.is;ieinputPath ,s.is;ieinpu tPath
To rename a file that alrea0y exits use'
+!!ean rena(ed X ,s.rena(einputPath8 ,s .rena(einputPath8 ne) PathY!dQ,ie.t0tY PathY!dQ ,ie.t0tY
RES#$T'
5hus the pr!'ra( t! use the >ad!!p ;ie Syste( API t! interact )ith it is success,uy e0ecuted.
Ex No' (
+RITE % +"RD!"#NT PR"R%M T" DEM"NSTR%TE THE #SE "F M%P %ND RED#!E T%SKS
D%TE'
"&-E!TIE'
5! *!rd c!unt pr!'ra( t! de(!nstrate the use !, Map and Reduce tas6s PR"!ED#RE' STEPS'
$. AnayD;S and (!vin' the data int! Input path :. E0ecutin' the pr!'ra( Sample Program'
i(p!rt 9ava.i!.IOE0cepti!n i(p!rt 9ava.uti.Strin'5!6eni
22had!!p supp!rted data types private ,ina static Int*rita+e !ne X ne) Int*rita+e$ private 5e0t 5e0t )!rd X ne) 5e0t 5e0t 22(ap (eth!d that per,!r(s the t!6eni
22purp!ses 1!+ 9!+ X ne) 1!+c!n,8 Y)!rd c!untY 9!+.set1arByCass*!rdC!unt.cass 22 Speci,y vari!us 9!+ speci,ic para(eters 9!+.setMapperCass5!6eni
+i'data&!cah!st@2h!(e2+i'data2D!)n!ads +i'data&!cah!st@2h !(e2+i'data2D!)n!ads2(rc!de2src_ 2(rc!de2src_ cd ..2+uid2 +i'data&!cah!st@2h!(e2+i'data2D!)n!ads +i'data&!cah!st@2h !(e2+i'data2D!)n!ads2(rc!de2+uid_ 2(rc!de2+uid_ 9ar cv, )c.9ar . Starting the DFS 1if not running alrea0y2
+i'data&!cah!st@2h!(e2+i'data2D!)n!ads +i'data&!cah!st@2h !(e2+i'data2D!)n!ads2had!!p-.4.$_s+in2s 2had!!p-.4.$_s+in2startd,s.sh tartd,s.sh !reating Input path in HDFS an0 mo)ing the 0ata into Input path
+i'data&!cah!st@2h!(e2+i'data2D!)n!ads +i'data&!cah!st@2h !(e2+i'data2D!)n!ads2had!!p-.4.$_+in 2had!!p-.4.$_+in2had!!p 2had!!p ,s (6dir2(rin +i'data&!cah!st@2h!(e2+i'data2D!)n!ads +i'data&!cah!st@2h !(e2+i'data2D!)n!ads2had!!p-.4.$_+in 2had!!p-.4.$_+in2had!!p 2had!!p ,s c!py;r!(L!ca 2h!(e2+i'data2D!)n!ads2(rc!de2(rsa(pedata2Z hd,s@22!cah!st@%###2(rin Executing the program
+i'data&!cah!st@2h!(e2+i'data2D!)n!ads +i'data&!cah!st@2h !(e2+i'data2D!)n!ads2had!!p-.4.$_+in 2had!!p-.4.$_+in2had!!p 2had!!p 9ar 2h!(e2+i'data2D!)n!ads2(rc!de2+uid2)c.9ar +i'data.*!rdC!unt 2(rin 2(r!ut$
RES#$T'
5hus 5h us the the *!rd !rd c!un c!untt pr!' pr!'ra ra( ( t! use use Map Map and and redu reduce ce tas6 tas6ss is de(! de(!ns nstr trat ated ed success,uy.
