i
Copyright Copyright © 2013 Dr. Martin Jones
This work is licensed licensed under a Creative Commons AttributionAttribution-NonCommer NonCommercialcialShareAlike 3.0 Unported License. License .
For more information, visit http://pythonforbioogists.!om "et in #$ "erif an% Source Code Pro
ii
About the author Martin starte% his programming !areer by earning #er %&ring the !o&rse of his #hD in evo&tionary bioogy, an% starte% tea!hing other peope to program soon after. "in!e then he has ta&ght intro%&!tory intro%&!tor y programming to h&n%re%s of bioogists, from &n%ergra%&ates to #'s, an% has maintaine% a phiosophy that programming !o&rses m&st be frien%y, approa!habe, approa!habe, an% pra!ti!a. Martin has ta&ght intro%&!tory programming as part of the (ioinformati!s M"! !o&rse at )%inb&rgh *niversity for the past five years, an% is !&rrenty +e!t&rer in (ioinformati!s.
iii
Preface e!ome to #ython for (ioogists. (efore yo& rea% any f&rther, ma-e s&re that this is the most re!ent version of the boo-. #ython for (ioogists is being !ontin&ay &p%ate% an% improve% to ta-e into a!!o&nt !orre!tions, amen%ments an% !hanges to #ython itsef, so its important that yo& are rea%ing the most &pto%ate version. $his fie is revision n&mber 189. $he n&mber of the most re!ent revision !an aays be fo&n% at: http://pythonforbioogists.!om/in%e.php/version/ 'f the revision n&mber iste% at the *+ is higher than the one in bo%, then this is an o&tof%ate o&tof%ate !opy, an% yo& nee% to %onoa% the atest version from http://pythonforbioogists.!om o& noti!e from the !opyright page that the !ontents of this boo- are i!ense% &n%er a Creative Commons 4ttrib&tion 4ttrib&tion "hare4i-e "hare 4i-e i!ense. $his means that yo&re free to %o hat yo& i-e ith it 5 !opy !opy it, emai it to yo&r frien%s, apaper yo&r ab ith it 5 as ong as yo& -eep the attrib&tion. o& !an aso mo%ify it, as ong as yo& i!ense yo&r mo%ifi!ation &n%er the same terms. $he ony thing that the i!ense %oesnt ao is !ommer!ia &se 5 if yo&% i-e to &se the !ontents of this !o&rse for for !ommer!ia p&rposes, get in to&!h ith me at martin6pythonforbioogists.!om
7appy programming8
iv
Table of Contents About the author » ii Preface » iii
1: Introduction and environment
1
Why have a programming book for biologists? » 1 Why Python? » 2 How to use this book » 5 !ercises an" solutions » # $etting in touch » % &etting up your environment » % 'e!t e"itors » 11 (ea"ing the "ocumentation » 12
2: Printing and manipulating text
13
Why are we so intereste" in working with te!t? » 1) Printing a message to the screen » 1* +uotes are important » 15 ,se comments to annotate your co"e » 1 rror messages an" "ebugging » 1% Printing special characters » 21 &toring strings in variables » 21 'ools for manipulating strings » 2* (ecap » )* !ercises » )&olutions » ).
3: Reading and riting files Why are we so intereste" in working with files? » 52 (ea"ing te!t from a file » 5) /iles0 contents an" file names » 55 ealing with newlines » 5# issing files » -3
!2
v
Writing te!t to files » -3 4losing files » -) Paths an" fol"ers » -) (ecap » -* !ercises » -5 &olutions » -#
": #ists and loops
$"
Why "o we nee" lists an" loops? » #* 4reating lists an" retrieving elements » #Working with list elements » ## Writing a loop » #. n"entation errors » %2 ,sing a string as a list » %) &plitting a string to make a list » %* terating over lines in a file » %* 6ooping with ranges » %5 (ecap » %# !ercises » %. &olutions » .3
!: %riting our on functions Why "o we want to write our own functions? » . . efining a function » 133 4alling an" improving our function » 13) ncapsulation with functions » 135 /unctions "on7t always have to take an argument » 13 /unctions "on7t always have to return a value » 13% /unctions can be calle" with name" arguments » 13% /unction arguments can have "efaults » 113 'esting functions » 111 (ecap » 11) !ercises » 115 &olutions » 11-
99
vi
&: Conditional tests
121
Programs nee" to make "ecisions » 121 4on"itions0 'rue an" /alse » 121 if statements » 12* else statements » 125 elif statements » 12while loops » 12% 8uil"ing up comple! con"itions » 12% Writing true9false functions » 1)3 (ecap » 1)1 !ercises » 1)) &olutions » 1)5
$: Regular expressions
1"1
'he importance of patterns in biology » 1*1 o"ules in Python » 1*) (aw strings » 1** &earching for a pattern in a string » 1*5 !tracting the part of the string that matche" » 153 $etting the position of a match » 152 &plitting a string using a regular e!pression » 15) /in"ing multiple matches » 15* (ecap » 155 !ercises » 15# &olutions » 15%
8: 'ictionaries
1&8
&toring paire" "ata » 1-% 4reating a "ictionary » 1#) terating over a "ictionary » 1#. (ecap » 1%2 !ercises » 1%) &olutions » 1%*
9: (iles) programs) and user input
19!
vii
/ile contents an" manipulation » 1.5 8asic file manipulation » 1. eleting files an" fol"ers » 1.% 6isting fol"er contents » 1.% (unning e!ternal programs » 1.. (unning a program » 233 &aving program output » 231 ,ser input makes our programs more fle!ible » 231 nteractive user input » 23) 4omman" line arguments » 23* (ecap » 235 !ercises » 23# &olutions » 23%
1
Chapter 1: 'ntro%&!tion an% environment
1:
Introduction and environment
Why have a programming book for biologists? 'f yo&re rea%ing this boo-, then yo& probaby %ont nee% to be !onvin!e% that programming is be!oming an in!reasingy essentia part of the too -it for bioogists of a types. o& might, hoever, nee% to be !onvin!e% that a boo- i-e this one, %eveope% espe!iay for bioogists, !an %o a better 9ob of tea!hing yo& to program than a generap&rpose intro%&!tory programming boo-. 7ere are a fe of the reason hy ' thin- that is the !ase. 4 bioogyspe!ifi! programming boo- aos &s to &se eampes an% eer!ises that &se bioogi!a probems. $his serves to important p&rposes: firsty, it provi%es motivation an% %emonstrates the types of probems that programming !an hep to sove. )perien!e has shon that beginners ma-e m&!h better progress hen they are motivate% by the tho&ght of ho the programs they rite i ma-e their ife easier8 "e!on%y, by &sing bioogi!a eampes, the !o%e an% eer!ises thro&gho&t the boo- !an form a ibrary of &sef& !o%e snippets, hi!h e !an refer ba!- to hen e ant to sove reaife probems. 'n bioogy, as in a fie%s of programming, the same probems ten% to re!&r time an% time again, so its very &sef& to have this !oe!tion of eampes to a!t as a referen!e 5 something thats not possibe ith a generap&rpose programming boo-. 4 bioogyspe!ifi! programming boo- !an aso !on!entrate on the feat&res of the ang&age that are most &sef& to bioogists. 4 ang&age i-e #ython has many feat&res an% in the !o&rse of earning it e inevitaby have to !on!entrate on some an% miss others o&t. $he set of feat&res hi!h are important to &s in bioogy are sighty %ifferent to those hi!h are most &sef& for generap&rpose programming 5 for eampe, e are m&!h more intereste% in manip&ating tet in!&%ing things i-e D;4 an% protein se<&en!es= than the average programmer. 4so, there are severa feat&res of #ython that o&% not normay be %is!&sse% in an intro%&!tory
2
Chapter 1: 'ntro%&!tion an% environment
programming boo-, b&t hi!h are very &sef& to bioogists for eampe, reg&ar epressions an% s&bpro!esses=. 7aving a bioogyspe!ifi! tetboo- aos &s to in!&%e these feat&res, aong ith epanations of hy they are parti!&ary &sef& to &s. 4 reate% point is that a tetboo- ritten 9&st for bioogists aos &s to intro%&!e feat&res in a ay that aos &s to start riting &sef& programs right aay. e !an %o this by ta-ing into a!!o&nt the sorts of probems that repeate%y !rop &p in bioogy, an% prioritising the feat&res that are best at soving them. $his boo- has been %esigne% so that yo& sho&% be abe to start riting sma b&t &sef& programs &sing ony the toos in the first !o&pe of !hapters.
Why Python? +et me start this se!tion ith the fooing statement: programming ang&ages are overrate%. hat ' mean by that is that peope ho are ne to programming ten% to orry far too m&!h abo&t hat ang&age to earn. $he !hoi!e of programming ang&age %oes matter, of !o&rse, b&t it matters far ess than peope thin- it %oes. $o p&t it another ays, !hoosing the >rong> programming ang&age is very &ni-ey to mean the %ifferen!e beteen fai&re an% s&!!ess hen earning. ?ther fa!tors motivation, having time to %evote to earning, hepf& !oeag&es= are far more important, yet re!eive ess attention. $he reason that peope pa!e so m&!h eight on the :what language shoul" learn?: <&estion is that its a big, obvio&s <&estion, an% its not %iffi!&t to fin% peope ho i give yo& strong opinions on the s&b9e!t. 'ts aso the first big <&estion that beginners have to anser on!e theyve %e!i%e% to earn programming, so it ass&mes a great %ea of importan!e in their min%s. $here are three main reasons hy !hoi!e of programming ang&age is not as important as most peope thin- it is. Firsty, neary everybo%y ho spen%s any signifi!ant amo&nt of time programming as part of their 9ob i event&ay en% &p
3
Chapter 1: 'ntro%&!tion an% environment
&sing m&tipe ang&ages. #arty this is 9&st %on to the simpe !onstraints of vario&s ang&ages 5 if yo& ant to rite a eb appi!ation yo& probaby %o it in Javas!ript, if yo& ant to rite a graphi!a &ser interfa!e yo& probaby &se something i-e Java, an% if yo& ant to rite oeve agorithms yo& probaby &se C. "e!on%y, earning a first programming ang&age gets yo& @0A of the ay toar%s earning a se!on%, thir%, an% fo&rth one. +earning to thin- i-e a programmer in the ay that yo& brea- %on !ompe tas-s into simpe ones is a s-i that !&ts a!ross a ang&ages 5 so if yo& spen% a fe months earning #ython an% then %is!over that yo& reay nee% to rite in C, yo&r time ont have been aste% as yo& be abe to pi!- it &p m&!h <&i!-er. $hir%y, the -in%s of probems that e ant to sove in bioogy are generay amenabe to being sove% in any ang&age, even tho&gh %ifferent programming ang&ages are goo% at %ifferent things. 'n other or%s, as a beginner, yo&r !hoi!e of ang&age is vanishingy &ni-ey to prevent yo& from soving the probems that yo& nee% to sove. 7aving sai% a that, hen earning to program e "o nee% to pi!- a ang&age to or- in, so e might as e pi!- one thats going to ma-e the 9ob easier. #ython is s&!h a ang&age for a n&mber of reasons: •
't has a mosty!onsistent synta, so yo& !an generay earn one ay of %oing things an% then appy it in m&tipe pa!es
•
't has a sensibe set of b&itin ibraries for %oing ots of !ommon tas-s
•
't is %esigne% in s&!h a ay that theres an obvio&s ay of %oing most things
•
'ts one of the most i%ey&se% ang&ages in the or%, an% theres a ot of a%vi!e, %o!&mentation an% t&torias avaiabe on the eb
•
'ts %esigne% in a ay that ets yo& start to rite &sef& programs as soon as possibe
B
Chapter 1: 'ntro%&!tion an% environment •
'ts &se of in%entation, hie annoying to peope ho arent &se% to it, is great for beginners as it enfor!es a !ertain amo&nt of rea%abiity
#ython aso has a !o&pe of points to re!ommen% it to bioogists an% s!ientists spe!ifi!ay: •
'ts i%ey &se% in the s!ientifi! !omm&nity
•
't has a !o&pe of very e%esigne% ibraries for %oing !ompe s!ientifi! !omp&ting atho&gh e ont en!o&nter them in this boo-=
•
't en% itsef e to being integrate% ith other, eisting toos
•
't has feat&res hi!h ma-e it easy to manip&ate strings of !hara!ters for eampe, strings of D;4 bases an% protein amino a!i% resi%&es, hi!h e as bioogists are parti!&ary fon% of=
P*t+on vs, Perl For bioogists, the <&estion :what language shoul" learn: often reay !omes %on to the <&estion :shoul" learn Perl or Python?: , so ets anser it hea% on. #er an% #ython are both perfe!ty goo% ang&ages for soving a i%e variety of bioogi!a probems. 7oever, after etensive eperien!e tea!hing both #er an% #ython to bioogists, 've !ome the !on!&sion that #ython is an easier ang&age to earn by virt&e of being more consistent an% more readable. 4n important thing to &n%erstan% abo&t #er an% #ython is that they are incre"ibly simiar %espite the fa!t that they oo- very %ifferent=, so the point above abo&t earning a se!on% ang&age appies %o&by. Many #ython an% #er feat&res have a onetoone !orrespon%en!e, an% so earning #er after earning #ython i be reativey easy 5 m&!h easier than, for eampe, moving to Java or C.
Chapter 1: 'ntro%&!tion an% environment
How to use this book #rogramming boo-s generay fa into to !ategories referen!etype boo-s, hi!h are %esigne% for oo-ing &p spe!ifi! bits of information, an% t&toriatype boo-s, hi!h are %esigne% to be rea% !overto!over. $his boo- is an eampe of the atter 5 !o%e sampes in ater !hapters often &se materia from previo&s ones, so yo& nee% to ma-e s&re yo& rea% the !hapters in or%er. )er!ises or eampes from one !hapter are sometimes &se% to i&strate the nee% for feat&res that are intro%&!e% in the net. $here are a n&mber of f&n%amenta programming !on!epts that are reevant to materia in m&tipe %ifferent !hapters. 'n this boo-, rather than intro%&!e these !on!epts a in one go, 've trie% to epain them as they be!ome ne!essary. $his res&ts in a ten%en!y for earier !hapters to be onger than ater ones, as they invove the intro%&!tion of more ne !on!epts. 4 !ertain amo&nt of 9argon is ne!essary if e ant to ta- abo&t programs an% programming !on!epts. 've trie% to %efine ea!h ne te!hni!a term at the point here its intro%&!e%, an% then &se it thereafter ith o!!asiona remin%ers of the meaning. Chapters ten% to foo a pre%i!tabe str&!t&re. $hey generay start ith a fe paragraphs o&tining the motivation behin% the feat&res that it i !over 5 hy %o they eist, hat probems %o they ao &s to sove, an% hy are they &sef& in bioogy spe!ifi!ayE $hese are fooe% by the main bo%y of the !hapter in hi!h e %is!&ss the reevant feat&res an% ho to &se them. $he ength of the !hapters varies <&ite a ot 5 sometimes e ant to !over a topi! briefy, other times e nee% more %epth. $his se!tion en%s ith a brief re!ap o&tining hat e have earne%, fooe% by eer!ises an% so&tions more on that topi! beo=. 4 !o&pe of notes on typography: bold t*pe is &se% to emphasie important points an% italics for te!hni!a terms an% fie names. here !o%e is mie% in ith norma tet its ritten in a mono-spaced font like this . ?!!asionay there are
G
Chapter 1: 'ntro%&!tion an% environment
footnotes1 to provi%e a%%itiona information that is interesting to -no b&t not !r&!ia to &n%erstan%ing, or to give in-s to eb pages. )ampe !o%e is highighte% ith a soi% bor%er: Some example code goes here
an% eampe o&tp&t i.e. hat e see on the s!reen hen e r&n the !o%e= is highighte% ith a %otte% bor%er: Some output goes here
?ften e ant to oo- at the !o%e an% the o&tp&t it pro%&!es together. 'n these sit&ations, yo& see a re%bor%ere% !o%e bo!- fooe% imme%iatey by a b&e bor%ere% o&tp&t bo!-. "ometimes its ne!essary to refer in the tet to in%ivi%&a ines of !o%e or o&tp&t, in hi!h !ase 've &se% ine n&mberings on the eft: 1
first line 2 second line 3 third line
?ther bo!-s of tet &s&ay fie !ontents or type% !omman% ines= %ont have any -in% of bor%er an% oo- i-e this: contents of a file
1
+i-e this.
H
Chapter 1: 'ntro%&!tion an% environment
!ercises an" solutions $he fina part of ea!h !hapter is a set of eer!ises an% so&tions. $he n&mber an% !ompeity of eer!ises %iffer greaty beteen !hapters %epen%ing on the nat&re of the materia. 4s a r&e, eary !hapters have a arge n&mber of simpe eer!ises, hie ater !hapters have a sma n&mber of more !ompe ones. Many of the eer!ise probems are ritten in a %eiberatey vag&e manner an% the ea!t %etais of ho the so&tions or- is &p to yo& very m&!h i-e reaife programming8= o& !an aays oo- at the so&tions to see one possibe ay of ta!-ing the probem, b&t there are often m&tipe vai% approa!hes. ' strongy re!ommen% that yo& try ta!-ing the eer!ises yo&rsef before rea%ing the so&tions there reay is no s&bstit&te for pra!ti!a eperien!e hen earning to program. ' aso en!o&rage yo& to a%opt an attit&%e of !&rio&s eperimentation hen or-ing on the eer!ises 5 if yo& fin% yo&rsef on%ering if a parti!&ar variation on a probem is sovabe, or if yo& re!ognie a !oseyreate% probem from yo&r on or-, try soving it8 Contin&o&s eperimentation is a -ey part of %eveoping as a programmer, an% the <&i!-est ay to fin% o&t hat a parti!&ar f&n!tion or feat&re i %o is to try it. $he eampe so&tions to eer!ises are ritten in a %ifferent ay to most programming tetboo-s: rather than simpy present the finishe% so&tion, ' have o&tine% the tho&ght pro!esses invove% in soving the eer!ises an% shon ho the so&tion is b&it &p stepbystep. 7opef&y this approa!h i give yo& an insight into the probemsoving min%set that programming re<&ires. 'ts probaby a goo% i%ea to rea% thro&gh the so&tions even if yo& s&!!essf&y sove the eer!ise probems yo&rsef, as they sometimes s&ggest an approa!h that is not imme%iatey obvio&s.
I
Chapter 1: 'ntro%&!tion an% environment
$etting in touch ?ne of the most !onvin!ing arg&ments for presenting a !o&rse i-e this one in the form of an eboo- is that it !an be !ontin&ay &p%ate% an% tea-e% base% on rea%er fee%ba!-. "o, if yo& fin% anything that is har% to &n%erstan%, or yo& thin- may !ontain an error, pease get in to&!h 5 9&st %rop me an emai at martin6pythonforbioogists.!om an% ' promise to get ba!- to yo&.
&etting up your environment 4 that yo& nee% in or%er to foo the eampes an% eer!ises in this boo- is a stan%ar% #ython instaation an% a tet e%itor. 4 the !o%e in this boo- i r&n on either +in&, Ma! or in%os ma!hines. $he sight %ifferen!es beteen operating systems are epaine% in the tet mosty in !hapter @=. 'f yo& have a !hoi!e of operating systems on hi!h to earn #ython, ' re!ommen% +in&, Ma! ?" an% in%os in that or%er, simpy be!a&se the *;'base% operating systems +in& an% ?"= are more amenabe to programming in genera.
Installing P*t+on $he pro!ess of instaing #ython %epen%s on the type of !omp&ter yo&re r&nning on. 'f yo&re r&nning a mainstream +in& %istrib&tion i-e *b&nt&, #ython is probaby area%y instae%. $o fin% o&t, open a termina an% type python
'f yo& see some o&tp&t aong these ines: Python 2.7.3 (default, Apr 10 2013, 0!13!1"# $%&& '.7.2 on linux2 )ype *help*, *copyright*, *credits* or *license* for more information. +++
@
Chapter 1: 'ntro%&!tion an% environment
$hen yo& are rea%y to go. 'f yo&r +in& instaation %oesnt area%y have #ython instae%, try instaing it ith yo&r pa!-age manager the !omman% i probaby be either sudo apt-get install python or sudo yum install python =. 'f this %oesnt or-, then %onoa% the pa!-age from the #ython %onoa% page 1. $he offi!ia #ython ebsite has instaation instr&!tions for Ma! 2 an% in%os 3 !omp&ters as e these are i-ey to be the most &pto%ate instr&!tions, so foo them !osey.
Running P*t+on programs 4 #ython program is 9&st a norma tet fie that !ontains #ython !o%e. $o r&n it e m&st first open &p a !omman% ine. ?n +in& an% Ma! !omp&ters, the appi!ation to %o this i be !ae% something aong the ines of >termina>. ?n in%os, it is -non as >!omman% prompt>. $o r&n a #ython program, e 9&st type the path to the #ython ee!&tabe fooe% by the name of the fie that !ontains the !o%e e ant to r&n B. ?n a +in& or Ma! ma!hine, the path i be something i-e: usrlocal-inpython
?n in%os, it i be something i-e: c!Python27python
1 2 3 B
http://.python.org/getit/ http://.python.org/getit/ma!/ http://.python.org/getit/in%os/ hen e refer to >a #ython program> in this boo-, e are &s&ay ta-ing abo&t the tet fie that ho%s the !o%e.
10
Chapter 1: 'ntro%&!tion an% environment
$o r&n a #ython program, its generay easiest to be in the same fo%er as it. (y !onvention, #ython programs are given the etension .py, so to r&n a program !ae% test.py, e 9&st type: usrlocal-inpython test.py
$here are a !o&pe of tri!-s that !an be &sef& hen eperimenting ith programs 1. Firsty, yo& !an r&n #ython in an intera!tive or >she>= mo%e by r&nning it itho&t the name of a program fie. $his aos yo& to type in%ivi%&a statements an% see the res&t straight aay. "e!on%y, yo& !an r&n #ython ith the -i option, hi!h i !a&se it to r&n yo&r program an% t+en enter intera!tive mo%e. $his !an be han%y if yo& ant to eamine the state of variabes after yo&r !o%e has r&n.
P*t+on 2 vs, P*t+on 3 4s i <&i!-y be!ome !ear if yo& spen% any amo&nt of time on the offi!ia #ython ebsite, there are to versions of #ython !&rrenty avaiabe. $he #ython or% is, at the time of riting, in the mi%%e of a transition from version 2 to version 3. 4 %is!&ssion of the pros an% !ons of ea!h version is e beyon% the s!ope of this boo-2, b&t heres hat yo& nee% to -no: insta #ython 3 if possibe, b&t if yo& en% &p ith #ython 2, %ont orry 5 a the !o%e eampes in the boo- i or- ith both versions.
1 2
Dont orry if these to options ma-e no sense to yo& right no 5 they i %o so ater on in the boo-, on!e yo&ve earne% hat statements an% variabes a!t&ay are. o& might en!o&nter riting onine that ma-es the 2 to 3 !hangeover seem i-e a big %ea, an% it is 5 b&t ony for eisting, arge pro9e!ts. hen riting !o%e from s!rat!h, as yo& be %oing hen earning, yo&re &ni-ey to r&n into any probems.
11
Chapter 1: 'ntro%&!tion an% environment
'f yo&re going to &se #ython 2, there is 9&st one thing that yo& have to %o in or%er to ma-e some of the !o%e eampes or-: in!&%e this ine at the start of a yo&r programs: from //future// import diision
e ont go into the epanation behin% this ine, e!ept to say that its ne!essary in or%er to !orre!t a sma <&ir- ith the ay that #ython 2 han%es %ivision of n&mbers. Depen%ing on hat version yo& &se, yo& might see sight %ifferen!es beteen the o&tp&t in this boo- an% the o&tp&t yo& get hen yo& r&n the !o%e on yo&r !omp&ter. 've trie% to note these %ifferen!es in the tet here possibe.
'e!t e"itors "in!e a #ython program is 9&st a tet fie, yo& !an !reate an% e%it it ith any tet e%itor of yo&r !hoi!e. ;ote that by a tet e%itor ' don-t mean a or% pro!essor 5 %o not try to e%it #ython programs ith Mi!rosoft or%, +ibre?ffi!e riter, or simiar toos, as they ten% to insert spe!ia formatting mar-s that #ython !annot rea%. hen !hoosing a tet e%itor, there is one feat&re that is essentia1 to have, an% one hi!h is ni!e to have. $he essentia feat&re is something thats &s&ay !ae% tab emulation. $he effe!t of this feat&re at first seems <&ite o%% hen enabe%, it repa!es any tab !hara!ters that yo& type ith an e<&ivaent n&mber of spa!e !hara!ters &s&ay set to fo&r=. $he reason hy this is &sef& is %is!&sse% at ength in !hapter B, b&t heres a brief epanation: #ython is very f&ssy abo&t yo&r &se of tabs an% spa!es, an% &ness yo& are very %is!ipine% hen typing, its easy to en% &p ith a mit&re of tabs an% spa!es in yo&r programs. $his !a&ses very inf&riating probems, be!a&se they oo- the same to yo&, b&t not to #ython8 $ab em&ation
1
?K, so its not stri!ty essentia, b&t yo& i fin% ife m&!h easer if yo& have it.
12
Chapter 1: 'ntro%&!tion an% environment
fies the probem by ma-ing it effe!tivey impossibe for yo& to type a tab !hara!ter. $he feat&re that is ni!e to have is synta! highlighting . $his i appy %ifferent !oo&rs to %ifferent parts of yo&r #ython !o%e, an% !an hep yo& spot errors more easiy. e!ommen%e% tet e%itors are .otepad// for in%os 1, Text%rangler for Ma! ?"2, an% gedit for +in&3, a of hi!h are freey avaiabe. ?n the eb an% esehere yo& may see referen!es to #ython 'D)s. 'D) stan%s for 'ntegrate% Deveopment )nvironment, an% they typi!ay !ombine a tet e%itor ith a !oe!tion of other &sef& programming toos. hie they !an spee% &p %eveopment for eperien!e% programmers, theyre not a goo% i%ea for beginners as they !ompi!ate things, so ' %ont re!ommen% yo& &se them.
(ea"ing the "ocumentation #art of the tea!hing phiosophy that 've &se% in riting this boo- is that its better to intro%&!e a fe &sef& feat&res an% f&n!tions rather than overhem yo& ith a !omprehensive ist. $he best pa!e to go hen yo& %o ant a !ompete ist of the options avaiabe in #ython is the offi!ia %o!&mentation B hi!h, !ompare% to many ang&ages, is very rea%abe.
1 2 3 B
http://notepa%p&sp&s.org/ http://.barebones.!om/pro%&!ts/$etranger/ https://pro9e!ts.gnome.org/ge%it/ http://.python.org/%o!/
13
Chapter 2: #rinting an% manip&ating tet
2:
Printing and manipulating text
Why are we so intereste" in working with te!t? 1
?pen the first page of a boo- abo&t earning #ython , an% the !han!es are that the first eampes of !o%e yo& see invove numbers. $heres a goo% reason for that: n&mbers are generay simper to or- ith than tet 5 there are not too many things yo& !an %o ith them on!e yo&ve got basi! arithmeti! o&t of the ay= an% so they en% themseves e to eampes that are easy to &n%erstan%. 'ts aso a pretty safe bet that the average person rea%ing a programming boo- is %oing so be!a&se they nee% to %o some n&mber!r&n!hing. "o hat ma-es this boo- %ifferent 5 hy is this first !hapter abo&t tet rather than n&mbersE $he anser is that, as bioogists, e have a parti!&ar interest in %eaing ith tet rather than n&mbers tho&gh of !o&rse, e nee% to earn ho to manip&ate n&mbers too=. "pe!ifi!ay, ere intereste% in parti!&ar types of tet that e !a se;uences < the D;4 an% protein se<&en!es that !onstit&te the %ata that e %ea ith in bioogy. $here are other reasons that e have a greater interest in or-ing ith tet than the average novi!e programmer. 4s s!ientists, the programs that e rite often nee% to or- as part of a pipeine, aongsi%e other programs that have been ritten by other peope. $o %o this, e often nee% to rite !o%e that !an understand the o&tp&t from some other program e !a this parsing = or produce o&tp&t in a format that another program !an operate on. (oth of these tas-s re<&ire manip&ating tet. 've hinte% above that !omp&ters !onsi%er n&mbers an% tet to be %ifferent in some ay. $hats an important i%ea, an% one that e ret&rn to in more %etai ater. For no, ' ant to intro%&!e an important pie!e of 9argon 5 the or% string . "tring is 1
?r in%ee%, any other programming ang&age
1B
Chapter 2: #rinting an% manip&ating tet
the or% e &se to refer to a bit of tet in a !omp&ter program it 9&st means a string of !hara!ters=. From this point on e &se the or% string hen ere ta-ing abo&t !omp&ter !o%e, an% e reserve the or% se;uence for hen ere %is!&ssing bioogi!a se<&en!es i-e D;4 an% protein.
#rinting a message to the s!reen $he first thing ere going to earn is ho to print 1 a message to the s!reen. 7eres a ine of #ython !o%e that i !a&se a frien%y message to be printe%. L&i!remin%er: soi% ines in%i!ate #ython !o%e, %otte% ines in%i!ate o&tp&t. print(*ello orld*#
+ets ta-e a oo- at the vario&s bits of this ine of !o%e, an% give some of them names: $he hoe ine is !ae% a statement .
print is the name of a function= $he f&n!tion tes #ython, in vag&e terms, hat e ant to %o 5 in this !ase, e ant to print some tet. $he f&n!tion name is aays 2 fooe% by parentheses3. $he bits of tet insi%e the parentheses are !ae% the arguments to the f&n!tion. 'n this !ase, e 9&st have one arg&ment ater on e see eampes of f&n!tions that ta-e more than one arg&ment, in hi!h !ase the arg&ments are separate% by !ommas=.
1
2 3
hen e ta- abo&t printing tet insi%e a !omp&ter program, e are not ta-ing abo&t pro%&!ing a %o!&ment on a printer. $he or% >print> is &se% for any o!!asion hen o&r program o&tp&ts some tet 5 in this !ase, the o&tp&t is %ispaye% in yo&r termina. $his is not stri!ty tr&e, b&t its easier to 9&st foo this r&e than orry abo&t the e!eptions. $here are severa %ifferent types of bra!-ets in #ython, so for !arity e i aays refer to parentheses hen e mean these: =, s;uare brackets hen e mean these: N an% curly brackets hen e mean these: OP
1
Chapter 2: #rinting an% manip&ating tet
$he arg&ments te #ython hat e ant to %o more spe!ifi!ay 5 in this !ase, the arg&ment tes #ython ea!ty hat it is e ant to print: a frien%y greeting. 4ss&ming yo&ve fooe% the instr&!tions in !hapter 1 an% set &p yo&r #ython environment, type the ine of !o%e above into yo&r favo&rite tet e%itor, save it, an% r&n it. o& sho&% see a singe ine of o&tp&t i-e this: ello orld
L&otes are important 'n norma riting, e ony s&rro&n% a bit of tet in <&otes hen e ant to sho that they are being sai% by somebo%y. 'n #ython, hoever, strings are ala*s s&rro&n%e% by <&otes. $hat is ho #ython is abe to te the %ifferen!e beteen the instr&!tions i-e the f&n!tion name= an% the %ata the thing e ant to print=. e !an &se either singe or %o&be <&otes for strings 5 #ython i happiy a!!ept either. $he fooing to statements behave ea!ty the same: print(*ello orld*# print(ello orld#
+ets ta-e a oo- at the o&tp&t to prove it 1: ello orld ello orld
o& noti!e that the o&tp&t above %oesnt !ontain <&otes 5 they are part of the !o%e, not part of the string itsef. 'f e do ant to in!&%e <&otes in the o&tp&t, the easiest thing to %o2 is &se the other type of <&otes for s&rro&n%ing the string: 1 2
From this point on, ' ont te yo& to !reate a ne fie, enter the tet, an% r&n the program for ea!h eampe 5 ' i simpy sho yo& the o&tp&t 5 b&t ' en!o&rage yo& to try the eampes yo&rsef. $he aternative is to pa!e a ba!-sash !hara!ter Q= before the <&ote 5 this is !ae% escaping the <&ote an%
1G
Chapter 2: #rinting an% manip&ating tet
print(*She said, ello orld*# print(e said, *ello orld*#
$he above !o%e i give the fooing o&tp&t: She said, ello orld e said, *ello orld*
(e !aref& hen riting an% rea%ing !o%e that invoves <&otes 5 yo& have to ma-e s&re that the <&otes at the beginning an% en% of the string mat!h &p.
*se !omments to annotate yo&r !o%e ?!!asionay, e ant to rite some tet in a program that is for h&mans to rea%, rather than for the !omp&ter to ee!&te. e !a this type of ine a comment . $o in!&%e a !omment in yo&r so&r!e !o%e, start the ine ith a hash symbo 1: 4 this is a comment, it ill -e ignored -y the computer print(*&omments are ery useful5*#
o&re going to see a ot of !omments in the so&r!e !o%e eampes in this boo-, an% aso in the so&tions to the eer!ises. Comments are a very &sef& ay to %o!&ment yo&r !o%e, for a n&mber of reasons: •
1
o& !an p&t the epanation of hat a parti!&ar bit of !o%e %oes right net to the !o%e itsef. $his ma-es it m&!h easier to fin% the %o!&mentation for a ine of !o%e that is in the mi%%e of a arge program, itho&t having to sear!h thro&gh a separate %o!&ment.
i prevent #ython from trying to interpret it. $his symbo has many names 5 yo& might -no it as n&mber sign, po&n% sign, o!tothorpe, sharp from m&si!a notation=, !ross, or pigpen.
1H
Chapter 2: #rinting an% manip&ating tet •
•
(e!a&se the !omments are part of the so&r!e !o%e, they !an never get mie% &p or separate%. 'n other or%s, if yo& are oo-ing at the so&r!e !o%e for a parti!&ar program, then yo& a&tomati!ay have the %o!&mentation as e. 'n !ontrast, if yo& -eep the %o!&mentation in a separate fie, it !an easiy be!ome separate% from the !o%e. 7aving the !omments right net to the !o%e a!ts as a remin%er to &p%ate the %o!&mentation henever yo& !hange the !o%e. $he ony thing orse than &n%o!&mente% !o%e is !o%e ith o% %o!&mentation that is no onger a!!&rate8
Dont ma-e the mista-e, by the ay, of thin-ing that !omments are ony &sef& if yo& are panning on shoing yo&r !o%e to somebo%y ese. hen yo& start riting yo&r on !o%e, yo& i be amae% at ho <&i!-y yo& forget the p&rpose of a parti!&ar se!tion or statement. 'f yo& are or-ing on a so&tion to one of the eer!ises in this boo- on Fri%ay afternoon, then !ome ba!- to it on Mon%ay morning, it i probaby ta-e yo& <&ite a hie to pi!- &p here yo& eft off. Comments !an hep ith this probem by giving yo& hints abo&t the p&rpose of !o%e, meaning that yo& spen% ess time trying to &n%erstan% yo&r o% !o%e, th&s spee%ing &p yo&r progress. 4 si%e benefit is that riting a !omment for a bit of !o%e reinfor!es yo&r &n%erstan%ing at the time yo& are %oing it. 4 goo% habit to get into is riting a <&i!- oneine !omment above any ine of !o%e that %oes something interesting: 4 print a friendly greeting print(*ello orld*#
o& see this te!hni<&e &se% a ot in the !o%e eampes in this boo-, an% ' en!o&rage yo& to &se it for yo&r on !o%e as e.
1I
Chapter 2: #rinting an% manip&ating tet
)rror messages an% %eb&gging 't may seem %epressing eary in the boo- to be ta-ing abo&t errors8 7oever, its orth pointing o&t at this eary stage that computer programs almost never or0 correctl* t+e first time. #rogramming ang&ages are not i-e nat&ra ang&ages 5 they have a very stri!t set of r&es, an% if yo& brea- any of them, the !omp&ter i not attempt to g&ess hat yo& inten%e%, b&t instea% i stop r&nning an% present yo& ith an error message. o&re going to be seeing a ot of these error messages in yo&r programming !areer, so ets get &se% to them as soon as possibe.
(orgetting uotes 7eres one possibe error e !an ma-e hen printing a ine of o&tp&t 5 e !an forget to in!&%e the <&otes: print(ello orld#
$his is easiy %one, so ets ta-e a oo- at the o&tp&t e get if e try to r&n the above !o%e 1: 1 2 3 4 5
6 python error.py ile *error.py*, line 1 print(ello orld# 8 Syntax9rror! inalid syntax
1
$he o&tp&t that yo& see might be very sighty %ifferent from this, %epen%ing on a b&n!h of fa!tors i-e yo&r operating system an% the ea!t version of #ython yo& are &sing.
1@
Chapter 2: #rinting an% manip&ating tet
eferring to the ine n&mbers on the eft e !an see that the name of the #ython fie is error.py ine 1= an% that the error o!!&rs on the first ine of the fie ine 2=. #ythons best g&ess at the o!ation of the error is 9&st before the !ose parentheses ine 3=. Depen%ing on the type of error, this !an be rong by <&ite a bit, so %ont rey on it too m&!h8 $he type of error is a SyntaxError ine =, hi!h mean that #ython !ant &n%erstan% the !o%e 5 it brea-s the r&es in some ay in this !ase, the r&e that strings m&st be s&rro&n%e% by <&otation mar-s=. e see %ifferent types of errors ater in this boo-.
pelling mista0es hat happens if e missspe the name of the f&n!tionE: prin(*ello orld*#
e get a %ifferent type of error 5 a NameError – an% the error message is a bit more hepf&: 1 2 3 4 5
6 python error.py )race-ac: (most recent call last#! ile *error.py*, line 1, in ;module+ prin(*ello orld*#
20
Chapter 2: #rinting an% manip&ating tet
$his time, #ython %oesnt try to sho &s here on the ine the error o!!&rre%, it 9&st shos &s the hoe ine ine B=. $he error message tes &s hi!h or% #ython %oesnt &n%erstan% ine =, so in this !ase, its <&ite easy to fi.
plitting a statement over to lines hat if e ant to print some o&tp&t that spans m&tipe inesE For eampe, e ant to print the or% >7eo> on one ine an% then the or% >or%> on the net ine 5 i-e this: ello =orld
e might try p&tting a ne ine in the mi%%e of o&r string i-e this: print(*ello =orld*#
b&t that ont or- an% e get the fooing error message: 1 2 3 4 5
6 python error.py ile *error.py*, line 1 print(*ello 8 Syntax9rror! 9>? hile scanning string literal
#ython fin%s the error hen it gets to the en% of the first ine of !o%e ine 2 in the o&tp&t=. $he error message ine = is a bit more !rypti! than the others. >6 stan%s for )n% ?f +ine, an% string literal means a string in <&otes. "o to p&t this error message in pain )ngish: : starte" rea"ing a string in ;uotes0 an" got to the en" of the line before came to the closing ;uotation mark: 'f spitting the ine &p %oesnt or-, then ho %o e get the o&tp&t e ant.....E
21
Chapter 2: #rinting an% manip&ating tet
Printing special characters $he reason that the !o%e above %i%nt or- is that #ython got !onf&se% abo&t hether the ne ine as part of the string hi!h is hat e ante%= or part of the source co"e hi!h is ho it as a!t&ay interprete%=. hat e nee% is a ay to in!&%e a ne ine as part of a string, an% &!-iy for &s, #ython has 9&st s&!h a too b&it in. $o in!&%e a ne ine, e rite a ba!-sash fooe% by the etter n 5 #ython -nos that this is a spe!ia !hara!ter an% i interpret it a!!or%ingy. 7eres the !o%e hi!h prints >7eo or%> a!ross to ines: 4 ho to include a ne line in the middle of a string print(*ellonorld*#
;oti!e that theres no nee% for a spa!e before or after the ne ine. $here are a fe other &sef& spe!ia !hara!ters as e, a of hi!h !onsist of a ba!-sash fooe% by a etter. $he ony ones hi!h yo& are i-ey to nee% for the eer!ises in this boo- are the tab !hara!ter \t= an% the carriage return !hara!ter \r=. $he tab !hara!ter !an sometimes be &sef& hen riting a program that i pro%&!e a ot of o&tp&t. $he !arriage ret&rn !hara!ter or-s a bit i-e a ne ine in that it p&ts the !&rsor ba!- to the start of the ine, b&t %oesnt a!t&ay start a ne ine, so yo& !an &se it to overrite o&tp&t 5 this is sometimes &sef& for ong r&nning programs.
&toring strings in variables ?K, eve been paying aro&n% ith the print f&n!tion for a hie ets intro%&!e something ne. e !an ta-e a string an% assign a name to it &sing an e<&as sign 5 e !a this a variable: 4 store a short @
22
Chapter 2: #rinting an% manip&ating tet
$he variabe my_dna no points to the string !"#C#"!. e !a this assigning a variabe, an% on!e eve %one it, e !an &se the variabe name instea% of the string itsef 5 for eampe, e !an &se it in a print statement1: 4 store a short @
;oti!e that hen e &se the variabe in a print statement, e %ont nee% any <&otation mar-s 5 the <&otes are part of the string, so they are area%y >b&it in> to the variabe my_dna. e !an !hange the va&e of a variabe as many times as e i-e on!e eve !reate% it: my/dna B *A)%&%)A* print(my/dna# 4 change the alue of my/dna my/dna B *)%%)&&A*
7eres a very important point that trips many beginners &p: variabe names are arbitrar* 5 that means that e !an pi!- +atever e li0e to be the name of a variabe. "o o&r !o%e above o&% or- in ea!ty the same ay if e pi!-e% a %ifferent variabe name: 4 store a short @
1
'f its not !ear hy this is &sef&, %ont orry 5 it i be!ome m&!h more apparent hen e oo- at some onger eampes.
23
Chapter 2: #rinting an% manip&ating tet
hat ma-es a goo% variabe nameE Reneray, its a goo% i%ea to &se a variabe name that gives &s a !&e as to hat the variabe refers to. 'n this eampe, my_dna is a goo% variabe name, be!a&se it tes &s that the !ontent of the variabe is a D;4 se<&en!e. Conversey, $anana is a ba% variabe name, be!a&se it %oesnt reay te &s anything abo&t the va&e thats store%. 4s yo& rea% thro&gh the !o%e eampes in this boo-, yo& get a better i%ea of hat !onstit&tes goo% an% ba% variabe names. $his i%ea 5 that names for things are arbitrary, an% !an be anything e i-e 5 is a theme that i o!!&r many times in this boo-, so its important to -eep it in min%. ?!!asionay yo& i see a variabe name that loo0s li0e it has some sort of reationship ith the va&e it points to: my/file B *my/file.txt*
b&t %ont be fooe%8 Sariabe names an% strings are separate things. ' sai% above that variabe names !an be anything e ant, b&t its a!t&ay not <&ite that simpe 5 there are some r&es e have to foo. e are ony aoe% to &se etters, n&mbers, an% &n%ers!ores, so e !ant have variabe names that !ontain o%% !hara!ters i-e T, U or A. e are not aoe% to start a name ith a n&mber tho&gh e !an &se n&mbers in the mi%%e or at the en% of a name=. Finay, e !ant &se a or% thats area%y b&it in to the #ython ang&age i-e >print>. 'ts aso important to remember that variabe names are !asesensitive, so my_dna% &'_(N!% &y_(N! and &y_(na are a separate variabes. $e!hni!ay this means that yo& !o&% &se a fo&r of those names in a #ython program to store %ifferent va&es, b&t pease %ont %o this 5 it is very easy to be!ome !onf&se% hen yo& &se very simiar variabe names.
2B
Chapter 2: #rinting an% manip&ating tet
'ools for manipulating strings ;o e -no ho to store an% print strings, e !an ta-e a oo- at a fe of the fa!iities that #ython has for manip&ating them. #ython has many b&itin toos for !arrying o&t !ommon operations, an% in this net se!tion e ta-e a oo- at them onebyone. 'n the eer!ises at the en% of this !hapter, e oo- at ho e !an &se m&tipe %ifferent toos together in or%er to !arry o&t more !ompe operations.
Concatenation e !an !on!atenate sti!- together= to strings &sing the V symbo 1. $his symbo i 9oin together the string on the eft ith the string on the right: my/dna B *AA))* C *%%&&* print(my/dna#
+ets ta-e a oo- at the o&tp&t: AA))%%&&
'n the above eampe, the things being !on!atenate% ere strings, b&t e !an aso &se variabes that point to strings: upstream B *AAA* my/dna B upstream C *A)%&* 4 my/dna is no *AAAA)%&*
1
e !a this the concatenation operator=
2
Chapter 2: #rinting an% manip&ating tet
e !an even 9oin m&tipe strings together in one go: upstream B *AAA* donstream B *%%%* my/dna B upstream C *A)%&* C donstream 4 my/dna is no *AAAA)%&%%%*
'ts important to reaie that the res&t of !on!atenating to strings together is itsef a string. "o its perfe!ty ?K to &se a !on!atenation insi%e a print statement: print(*ello* C * * C *orld*#
4s e see in the rest of the boo-, &sing one too insi%e another is <&ite a !ommon thing to %o in #ython.
(inding t+e lengt+ of a string 4nother &sef& b&itin too in #ython is the len f&n!tion len is short for ength=. J&st i-e the print f&n!tion, the len f&n!tion ta-es a singe arg&ment ta-e a <&i!- oo- ba!- at hen e ere %is!&ssing the print f&n!tion for a remin%er abo&t hat arg&ments are= hi!h is a string. 7oever, the behavio&r of the len f&n!tion is <&ite %ifferent. 'nstea% of o&tp&tting tet to the s!reen, len o&tp&ts a va&e that !an be store% 5 e !a this the return value. 'n other or%s, if e rite a program that &ses len to !a!&ate the ength of a string, the program i r&n b&t e ont see any o&tp&t: 4 this line doesnt produce any output len(*A)%&*#
'f e ant to a!t&ay &se the ret&rn va&e, e nee% to store it in a variabe, an% then %o something &sef& ith it i-e printing it=:
2G
Chapter 2: #rinting an% manip&ating tet
dna/length B len(*A%)&*# print(dna/length#
$heres another interesting thing abo&t the len f&n!tion: the res&t or return value= is not a string, its a n&mber. $his is a very important i%ea so 'm going to rite it o&t in bo%: P*t+on treats strings and numbers differentl*, e !an see that this is the !ase if e try to !on!atenate together a n&mber an% a string. Consi%er this short program hi!h !a!&ates the ength of a D;4 se<&en!e an% then prints a message teing &s the ength: 4 store the @
hen e try to r&n this program, e get the fooing error: 1 2 3 4 5
6 python error.py )race-ac: (most recent call last#! ile *error.py*, line ", in ;module+ print(*)he length of the @
$he error message ine = is short b&t informative: >cannot concatenate )str) and )int) o$*ects >. #ython is !ompaining that it %oesnt -no ho to !on!atenate a string hi!h it !as str for short= an% a n&mber hi!h it !as int 5 short for integer=. "trings an% n&mbers are eampes of types 5 %ifferent -in%s of information that !an eist insi%e a program.
2H
Chapter 2: #rinting an% manip&ating tet
7appiy, #ython has a b&itin so&tion 5 a f&n!tion !ae% str hi!h t&rns a n&mber1 into a string so that e !an print it. 7eres ho e !an mo%ify o&r program to &se it 5 've remove% the !omments from this version to ma-e it a bit more !ompa!t: my/dna B *A)%&%A%)* dna/length B len(my/dna# print(*)he length of the @
$he ony thing e have !hange% is that eve repa!e dna_length ith str+dna_length, insi%e the print statement2. ;oti!e that be!a&se ere &sing one f&n!tion str= insi%e another f&n!tion print=, o&r statement no en%s ith to !osing parentheses. $o finish o&r %is!&ssion of the str f&n!tion, heres a forma %es!ription of it, ith a the te!hni!a terms in itai!s:
str is a function hi!h ta-es one argument hose type is number =, an% returns a va&e hose type is string = representing that n&mber. 'f yo&re &ns&re abo&t the meanings of any of the or%s in itai!s, s-ip ba!- to the earier parts of this !hapter here e %is!&sse% them. *n%erstan%ing ho types or- is -ey to avoi%ing many of the fr&strations hi!h ne programmers typi!ay en!o&nter, so ma-e s&re the i%ea is !ear in yo&r min% before moving on ith the rest of this boo-.
C+anging case e !an !onvert a string to oer !ase by &sing a ne type of synta 5 a metho" that beongs to strings. 4 metho" is i-e a function, b&t instea% of being b&it in to the
1 2
?r a va&e of any nonstring type, b&t e !ome to that ater 'f yo& eperiment ith some of the !o%e here, yo& might %is!over that yo& !an aso print a n&mber %ire!ty itho&t &sing str 5 b&t ony if yo& %ont try to !on!atenate it.
2I
Chapter 2: #rinting an% manip&ating tet
#ython ang&age, it beongs to a parti!&ar type. $he metho% e are ta-ing abo&t here is !ae% loer, an% e say that it beongs to the string type. 7eres ho e &se it: my/dna B *A)%&* 4 print my/dna in loer case print(my/dna.loer(##
;oti!e ho &sing a metho% oo-s %ifferent to &sing a f&n!tion. hen e &se a f&n!tion i-e print or len, e rite the f&n!tion name first an% the arg&ments go in parentheses: print(*A)%&*# len(my/dna#
hen e &se a metho%, e rite the name of the variabe first, fooe% by a perio%, then the name of the metho%, then the metho% arg&ments in parentheses. For the eampe ere oo-ing at here, loer, there is no arg&ment, so the opening an% !osing parentheses are right net to ea!h other. 'ts important to noti!e that the loer metho% %oes not a!t&ay !hange the variabe instea% it ret&rns a !opy of the variabe in oer !ase. e !an prove that it or-s this ay by printing the variabe before an% after r&nning loer. 7eres the !o%e to %o so: my/dna B *A)%&* 4 print the aria-le print(*-efore! * C my/dna# 4 run the loer method and store the result loercase/dna B my/dna.loer(# 4 print the aria-le again print(*after! * C my/dna#
an% heres the o&tp&t e get:
2@
Chapter 2: #rinting an% manip&ating tet
-efore! A)%& after! A)%&
J&st i-e the len f&n!tion, in or%er to a!t&ay %o anything &sef& ith the loer metho%, e nee% to store the res&t or print it right aay=. (e!a&se the loer metho% beongs to the string type, e !an ony &se it on variabes that are strings. 'f e try to &se it on a n&mber: n&mber: my/num-er B len(*A%)&*# 4 my/num-er is ' print(my/num-er.loer(##
e i get an error that that oo-s i-e this: Attri-ute9rror! int o-Dect has no attri-ute loer
$he error message is a bit !rypti!, b&t hopef&y yo& !an grasp the meaning: something that is a n&mber an int, or integer= %oes not have a loer metho%. $his is a goo% eampe of the importan!e of types in #ython !o%e: e can onl* onl* use met+ods on t+e t*pe t+at t+e* belong to . (efore e move on, ets ets 9&st mention that there is another metho% that t hat beongs to the string type !ae% upper 5 yo& yo& !an probaby g&ess hat it %oes8
Replacement 7eres another eampe of a &sef& metho% 7eres m etho% that beongs to the string type: replace. replace is sighty %ifferent from anything eve seen before 5 it ta-es to arg&ments both strings= an% ret&rns a !opy of the variabe here a o!!&rren!es o!!&rren! es of the first string are repa!e% by the se!on% string. $hats $hats <&ite a ong in%e% %es!ription, %es!ription, so here are a fe fe eampes to ma-e things things !earer:
30
Chapter 2: #rinting an% manip&ating tet
protein B *lspad:tn* 4 replace aline ith tyrosine print(protein.replace(**, *y*## 4 e can replace more than one character print(protein.replace(*ls*, *ymt*## 4 the original aria-le is not affected print(protein#
4n% this is the o&tp&t e get: ylspad:tny ymtpad:tn lspad:tn
e ta-e a oo- at more toos for !arrying o&t string repa!ement in !hapter H.
xtracting part of a string hat %o e %o if e have a ong string, b&t e ony ant a short portion of itE $his is -non as ta-ing a substring , an% it has its on notation in #ython. $o get a s&bstring, e foo foo the variabe name ith a pair of s<&are bra!-ets hi!h en!ose a start an% stop position, separate% by a !oon. 4gain, 4gain, this is probaby easier to vis&aie ith a !o&pe !o&pe of eampes 5 ets ets re&se o&r protein se<&en!e from before: before: protein B *lspad:tn* 4 print positions three to fie print(protein$3!# 4 positions start at Eero, not one print(protein$0!"# 4 if e use a stop position -eyond the end, its the same as using the end print(protein$0!"0#
an% heres the o&tp&t:
31
Chapter 2: #rinting an% manip&ating tet
pa lspad lspad:tn
$here are to important things to noti!e here. Firsty, e a!t&ay start !o&nting from position ero, rather than one 5 in other or%s, position 3 is a!t&ay the fo&rth !hara!ter1. $his epains hy the first first !hara!ter of the first ine of o&tp&t o&tp&t is p an% not s as yo& might thin-. "e!on%y, the positions are inclusive at the start, b&t exclusive at the stop. 'n other or%s, the epression protein/012 gives &s everything starting at the thir% !hara!ter, an% stopping 9&st before the fifth !hara!ter i.e. !hara!ters three an% fo&r=. 'f e 9&st give a singe n&mber in the s<&are bra!-ets, e e 9&st get a singe !hara!ter: protein B *lspad:tn* first/residue B protein$0
e earn a ot more abo&t this type of notation, an% hat e !an %o ith it, in !hapter B.
Counting and finding substrings 4 very !ommon 9ob in bioogy is to !o&nt the n&mber of times some pattern o!!&rs in a D;4 or protein se<&en!e. 'n !omp&ter !omp&ter programming terms, hat that transates to is !o&nting the n&mber of times a substring o!!&rs o!!&rs in a string . $he metho% that %oes the 9ob is !ae% count. 't ta-es a singe arg&ment hose type is string, an% ret&rns the n&mber of times that the arg&ment is fo&n% in the variabe. $he ret&rn type is a n&mber, so be !aref& abo&t ho yo& yo& &se it8
1
$his seems seems very very annoying annoying hen hen yo& yo& first en!o& en!o&nter nter it, it, b&t b&t e e see ater ater hy hy its its ne!essary ne!essary..
32
Chapter 2: #rinting an% manip&ating tet
+ets &se o&r protein +ets pr otein se<&en!e one ast time as an eampe. emember that e have to &se o&r o% frien% str to t&rn the !o&nts into strings so that e !an print them. 4so, noti!e noti!e that here ' have &se% a ban- ine to separate o&t the to bits of the program !a!&ating the !o&nts, an% printing them=. #ython is perfe!ty happy ith this 5 it 9&st 9&st ignores ban- ines, so its its fine to p&t them in in or%er to ma-e ma-e yo&r programs more rea%abe rea%abe for h&mans. protein B *lspad:tn* 4 count amino acid residues aline/count B protein.count(# lsp/count B protein.count(lsp# tryptophan/count B protein.count(# 4 no print the counts print(*alines! * C str(aline/count## print(*lsp! * C str(lsp/count## print(*tryptophans! * C str(tryptophan/count##
$he o&tp&t shos ho the !o&nt metho% behaves: alines! 2 leucines! 1 tryptophans! 0
4 !oseyreate% probem to !o&nting s&bstrings is fin%ing their o!ation. hat if instea% of !o&nting the n&mber of proine resi%&es in o&r protein se<&en!e e ant to -no here they areE $he find metho% i give &s the anser, at east for simpe !ases. find ta-es a singe string arg&ment, 9&st i-e count, an% ret&rns a n&mber hi!h is the position at hi!h that s&bstring first appears in the string in !omp&ting, e !a that the in"e! of the s&bstring=.
33
Chapter 2: #rinting an% manip&ating tet
emember that in #ython e start !o&nting from ero rather than one, so position 0 is the first !hara!ter, position B is the fifth !hara!ter, et!. 4 !o&pe of eampes: protein B *lspad:tn* print(str(protein.find(p### print(str(protein.find(:t### print(str(protein.find(###
4n% the o&tp&t: 3 " F1
;oti!e the behavio&r of fin% hen e as- it to o!ate a s&bstring that %oesnt eist 5 e get ba!- the anser -3. (oth count an% find have a pretty serio&s imitation: yo& !an ony sear!h for ea!t s&bstrings. 'f yo& nee% to !o&nt the n&mber of o!!&rren!es of a variabe protein motif, or fin% the position of a variabe trans!ription fa!tor bin%ing site, they i not hep yo&. $he hoe of !hapter H is %evote% to toos that !an %o those -in%s of 9obs. ?f the toos eve %is!&sse% in this se!tion, three 5 replace, count an% find 5 re<&ire at east to strings to or-, so be !aref& that yo& %ont get !onf&se% abo&t the or%er 5 remember that: my_dna.count+my_motif,
is not the same as: my/motif.count(my/dna#
3B
Chapter 2: #rinting an% manip&ating tet
"pitting &p a string into m&tipe bits 4n obvio&s <&estion hi!h bioogists often as- hen earning to program is >ho %o e spit a string e.g. a D;4 se<&en!e= into m&tipe pie!esE> $hats a !ommon 9ob in bioogy, b&t &nfort&natey e !ant %o it yet &sing the toos from this !hapter. e ta- abo&t vario&s %ifferent ays of spitting strings in !hapter B. ' mention it here 9&st to reass&re yo& that e i earn ho to %o it event&ay8
(ecap e starte% this !hapter ta-ing abo&t strings an% ho to or- ith them, b&t aong the ay e ha% to ta-e a ot of %iversions, a of hi!h ere ne!essary to &n%erstan% ho the %ifferent string toos or-. $han-f&y, that means that eve !overe% most of the n&ts an% bots of the #ython ang&age, hi!h i ma-e f&t&re !hapters go m&!h more smoothy. eve earne% abo&t some genera feat&res of the #ython programming ang&age i-e •
the %ifferen!e beteen functions, statements an% arguments
•
the importan!e of comments an% ho to &se them
•
ho to &se #ythons error messages to fi b&gs in o&r programs
•
ho to store values in variables
•
the ay that types or-, an% the importan!e of &n%erstan%ing them
•
the %ifferen!e beteen functions an% metho"s, an% ho to &se them both
4n% eve en!o&ntere% some toos that are spe!ifi!ay for or-ing ith strings: •
!on!atenation
•
%ifferent types of <&otes an% ho to &se them
•
spe!ia !hara!ters
3
Chapter 2: #rinting an% manip&ating tet •
!hanging the !ase of a string
•
fin%ing an% !o&nting s&bstrings
•
repa!ing bits of a string ith something ne
•
etra!ting bits of a string to ma-e a ne string
Many of the above topi!s i !rop &p again in f&t&re !hapters, an% i be %is!&sse% in more %etai, b&t yo& !an aays ret&rn to this !hapter if yo& ant to br&sh &p on the basi!s. $he eer!ises for this !hapter i ao yo& to pra!ti!e &sing the string manip&ation toos an% to be!ome famiiar ith them. $hey aso give yo& the !han!e to pra!ti!e b&i%er bigger programs by &sing the in%ivi%&a toos as b&i%ing bo!-s.
3G
Chapter 2: #rinting an% manip&ating tet
!ercises Reminder: the %es!riptions of the eer!ises are %eiberatey terse an% may be somehat ambig&o&s 9&st i-e re<&irements for programs yo& i rite in rea ife=. "ee the so&tions for in%epth %is!&ssions of the eer!ises.
Calculating 4T content 7eres a short D;4 se<&en!e: A&)%A)&%A))A&%)A)A%)A)))%&)A)&A)A&A)A)A)A)&%A)%&%))&A)
rite a program that i print o&t the 4$ !ontent of this D;4 se<&en!e. 7int: yo& !an &se norma mathemati!a symbos i-e a%% V=, s&btra!t =, m&tipy W=, %ivi%e /= an% parentheses to !arry o&t !a!&ations on n&mbers in #ython. Reminder: if yo&re &sing #ython 2 rather than #ython 3, in!&%e this ine at the top of yo&r program: from //future// import diision
Complementing '.4 7eres a short D;4 se<&en!e: A&)%A)&%A))A&%)A)A%)A)))%&)A)&A)A&A)A)A)A)&%A)%&%))&A)
rite a program that i print the !ompement of this se<&en!e.
3H
Chapter 2: #rinting an% manip&ating tet
Restriction fragment lengt+s 7eres a short D;4 se<&en!e: A&)%A)&%A))A&%)A)A%)A%AA))&)A)&A)A&A)A)A)A)&%A)%&%))&A)
$he se<&en!e !ontains a re!ognition site for the )!o' restri!tion enyme, hi!h !&ts at the motif RW44$$C the position of the !&t is in%i!ate% by an asteris-=. rite a program hi!h i !a!&ate the sie of the to fragments that i be pro%&!e% hen the D;4 se<&en!e is %igeste% ith )!o'.
plicing out introns) part one 7eres a short se!tion of genomi! D;4: 4$CR4$CR4$CR4$CR4C$R4C$4R$C4$4RC$4$RC4$R$4RC$4C$CR4$CR4$CR4 $CR4$CR4$CR4$CR4$CR4$CR4$C4$RC$4$C4$CR4$CR4$4$CR4$RC4$CR4C$ 4C$4$ 't !omprises to eons an% an intron. $he first eon r&ns from the start of the se<&en!e to the sitythir% !hara!ter, an% the se!on% eon r&ns from the ninety first !hara!ter to the en% of the se<&en!e. rite a program that i print 9&st the !o%ing regions of the D;4 se<&en!e.
plicing out introns) part to *sing the %ata from part one, rite a program that i !a!&ate hat per!entage of the D;4 se<&en!e is !o%ing. Reminder: if yo&re &sing #ython 2 rather than #ython 3, in!&%e this ine at the top of yo&r program: from //future// import diision
3I
Chapter 2: #rinting an% manip&ating tet
plicing out introns) part t+ree *sing the %ata from part one, rite a program that i print o&t the origina genomi! D;4 se<&en!e ith !o%ing bases in &pper!ase an% non!o%ing bases in oer!ase.
3@
Chapter 2: #rinting an% manip&ating tet
&olutions Calculating 4T content $his eer!ise is going to invove a mit&re of strings an% n&mbers. +ets remin% o&rseves of the form&a for !a!&ating 4$ !ontent: AT content =
A + T length
$here are three n&mbers e nee% to fig&re o&t: the n&mber of !s, the n&mber of "s, an% the ength of the se<&en!e. e -no that e !an get the ength of the se<&en!e &sing the len f&n!tion, an% e !an !o&nt the n&mber of !s an% "s &sing the count metho%. 7ere are a fe ines of !o%e that e thin- i !a!&ate the n&mbers e nee%: my/dna B *A&)%A)&%A))A&%)A)A%)A)))%&)A)&A)A&A)A)A)A)&%A)%&%))&A)* length B len(my/dna# a/count B my/dna.count(A# t/count B my/dna.count()#
4t this point, it seems sensibe to !he!- these ines before e go any f&rther. "o rather than %iving straight in an% %oing some !a!&ations, ets print o&t these n&mbers so that e !an eyeba them an% see if they oo- approimatey right. e have to remember to t&rn the n&mbers into strings &sing str so that e !an print them:
B0
Chapter 2: #rinting an% manip&ating tet
my/dna B *A&)%A)&%A))A&%)A)A%)A)))%&)A)&A)A&A)A)A)A)&%A)%&%))&A)* length B len(my/dna# a/count B my/dna.count(A# t/count B my/dna.count()# print(*length! * C str(length## print(*A count! * C str(a/count## print(*) count! * C str(t/count##
+ets ta-e a oo- at the o&tp&t from this program: length! ' A count! 1" ) count! 21
$hat oo-s abo&t right, b&t ho %o e -no if its ea!ty rightE e !o&% go thro&gh the se<&en!e man&ay base by base, an% verify that there are siteen 4s an% eighteen $s, b&t that %oesnt seem i-e a great &se of o&r time: aso, hat o&% e %o if the se<&en!e ere 1 -iobases rather than 1 basesE 4 better i%ea is to r&n the ea!t same !o%e ith a m&!h shorter test se<&en!e, to verify that it or-s before going ahea% an% r&nning it on the arger se<&en!e. 7eres a version that &ses a very short test se<&en!e ith one of ea!h of the fo&r bases: test/dna B *A)%&* length B len(test/dna# a/count B test/dna.count(A# t/count B test/dna.count()# print(*length! * C str(length## print(*A count! * C str(a/count## print(*) count! * C str(t/count##
an% heres the o&tp&t:
B1
Chapter 2: #rinting an% manip&ating tet
length! ' A count! 1 ) count! 1
)verything oo-s ?K 5 e !an probaby go ahea% an% r&n the !o%e on the ong se<&en!e. (&t ait e -no that the net step is going to invove %oing some !a!&ations &sing the n&mbers. 'f e sit!h ba!- to the ong se<&en!e no, then e be in the same position as e ere before 5 e en% &p ith an anser for the 4$ !ontent, b&t e ont -no if its the right one. 4 better pan is to sti!- ith the short test se<&en!e &nti eve ritten the hoe program, an% !he!- that e get the right anser for the 4$ !ontent e !an easiy see by gan!ing at the test se<&en!e that the 4$ !ontent is 0.=. 7ere goes 5 e &se the a%% an% %ivi%e symbos from the eer!ise hint: test/dna B *A)%&* length B len(test/dna# a/count B test/dna.count(A# t/count B test/dna.count()# at/content B a/count C t/count length print(*A) content is * C str(at/content##
$he o&tp&t from this program oo-s i-e this: A) content is 1.2
$hat %oesnt oo- right. +oo-ing ba!- at the !o%e e !an see hat has gone rong 5 in the !a!&ation, the %ivision has ta-en pre!e%en!e over the a%%ition, so hat e have a!t&ay !a!&ate% is: A +
T length
B2
Chapter 2: #rinting an% manip&ating tet
$o fi it, a e nee% to %o is a%% some parentheses aro&n% the a%%ition, so that the ine be!omes: at/content B (a/count C t/count# length
;o e get the !orre!t o&tp&t for the test se<&en!e: A) content is 0.
an% e !an go ahea% an% r&n the program &sing the onger se<&en!e, !onfi%ent that the !o%e is or-ing an% that the !a!&ations are !orre!t. 7eres the fina version: my/dna B *A&)%A)&%A))A&%)A)A%)A)))%&)A)&A)A&A)A)A)A)&%A)%&%))&A)* length B len(my/dna# a/count B my/dna.count(A# t/count B my/dna.count()# at/content B (a/count C t/count# length print(*A) content is * C str(at/content##
an% the fina o&tp&t: A) content is 0."G1G1G1G1G2
Complementing '.4 $his one seems pretty straightforar% 5 e nee% to ta-e o&r se<&en!e an% repa!e 4 ith $, $ ith 4, C ith R, an% R ith C. e have to ma-e fo&r separate !as to replace, an% &se the ret&rn va&e for ea!h on as the inp&t for the net tone. +ets try it:
B3
Chapter 2: #rinting an% manip&ating tet
my/dna B *A&)%A)&%A))A&%)A)A%)A)))%&)A)&A)A&A)A)A)A)&%A)%&%))&A)* 4 replace A ith ) replacement1 B my/dna.replace(A, )# 4 replace ) ith A replacement2 B replacement1.replace(), A# 4 replace & ith % replacement3 B replacement2.replace(&, %# 4 replace % ith & replacement' B replacement3.replace(%, &# 4 print the result of the final replacement print(replacement'#
hen e ta-e a oo- at the o&tp&t, hoever, something seems rong: A&A&AA&&AAAA&&AAAA&AAAAA&&AAA&AAA&AAAAAAAA&&AA&&&AA&AA
e !an see 9&st by oo-ing at the origina se<&en!e that the first etter is 4, so the first etter of the printe% se<&en!e sho&% be its !ompement, $. (&t instea% the first etter is 4. 'n fa!t, a of the bases in the printe% se<&en!e are either 4 or $. $his is %efinitey not hat e ant8 +ets try an% tra!- the probem %on by printing o&t a the interme%iate steps as e: my/dna B *A&)%A)&%A))A&%)A)A%)A)))%&)A)&A)A&A)A)A)A)&%A)%&%))&A)* replacement1 B my/dna.replace(A, )# print(replacement1# replacement2 B replacement1.replace(), A# print(replacement2# replacement3 B replacement2.replace(&, %# print(replacement3# replacement' B replacement3.replace(%, &# print(replacement'#
$he o&tp&t from this program ma-es it !ear hat the probem is:
BB
Chapter 2: #rinting an% manip&ating tet
)&)%))&%))))&%))))%)))))%&)))&)))&))))))))&%))%&%))&)) A&A%AA&%AAAA&%AAAA%AAAAA%&AAA&AAA&AAAAAAAA&%AA%&%AA&AA A%A%AA%%AAAA%%AAAA%AAAAA%%AAA%AAA%AAAAAAAA%%AA%%%AA%AA A&A&AA&&AAAA&&AAAA&AAAAA&&AAA&AAA&AAAAAAAA&&AA&&&AA&AA
$he first repa!ement the res&t of hi!h is shon in the first ine of the o&tp&t= or-s fine 5 a the 4s have been repa!e% ith $s for eampe, oo- at the first !hara!ter 5 its 4 in the origina se<&en!e an% $ in the first ine of the o&tp&t=. $he se!on% repa!ement is here it starts to go rong: a the $s are repa!e% by 4s, including t+ose t+at ere t+ere as a result of t+e first replacement. "o %&ring the first to repa!ements, the first !hara!ter is !hange% from 4 to $ an% then straight ba!- to 4 again. 7o are e going to get ro&n% this probemE ?ne option is to pi!- a temporary aphabet of fo&r etters an% %o ea!h repa!ement ti!e: my/dna B *A&)%A)&%A))A&%)A)A%)A)))%&)A)&A)A&A)A)A)A)&%A)%&%))&A)* replacement1 B my/dna.replace(A, # replacement2 B replacement1.replace(), H# replacement3 B replacement2.replace(&, I# replacement' B replacement3.replace(%, ?# replacement B replacement'.replace(, )# replacement" B replacement.replace(H, A# replacement7 B replacement".replace(I, %# replacementG B replacement7.replace(?, &# print(replacementG#
$his gets &s the res&t e are oo-ing for. 't avoi%s the probem ith the previo&s program by &sing another etter to stan% in for ea!h base hie the repa!ements are being %one. For eampe, 4 is first !onverte% to 7 an% then ater on 7 is !onverte% to $. 7eres a sighty more eegant ay of %oing it. e !an ta-e a%vantage of the fa!t that the replace metho% is !asesensitive, an% ma-e a the repa!e% bases oer
B
Chapter 2: #rinting an% manip&ating tet
!ase. $hen, on!e a the repa!ements have been !arrie% o&t, e !an simpy !a upper an% !hange the hoe se<&en!e ba!- to &pper !ase. +ets ta-e a oo- at ho this or-s: my/dna B *A&)%A)&%A))A&%)A)A%)A)))%&)A)&A)A&A)A)A)A)&%A)%&%))&A)* replacement1 B my/dna.replace(A, t# print(replacement1# replacement2 B replacement1.replace(), a# print(replacement2# replacement3 B replacement2.replace(&, g# print(replacement3# replacement' B replacement3.replace(%, c# print(replacement'# print(replacement'.upper(##
$he o&tp&t ets &s see ea!ty hats happening 5 noti!e that in this version of the program e print the fina string ti!e, on!e as it is an% then on!e !onverte% to &pper !ase: t&)%t)&%t))t&%)t)t%)t)))%&)t)&t)t&t)t)t)t)&%t)%&%))&t) t&a%ta&%taat&%atat%ataaa%&ata&tat&tatatata&%ta%&%aa&ta tga%tag%taatg%atat%ataaa%gatagtatgtatatatag%ta%g%aagta tgactagctaatgcatatcataaacgatagtatgtatatatagctacgcaagta )%A&)A%&)AA)%&A)A)&A)AAA&%A)A%)A)%)A)A)A)A%&)A&%&AA%)A
e !an see that as the program r&ns, ea!h base in t&rn is repa!e% by its !ompement in oer !ase. "in!e the net repa!ement is ony oo-ing for &pper !ase !hara!ters, bases %ont get !hange% ba!- as they %i% in the first version of o&r program.
BG
Chapter 2: #rinting an% manip&ating tet
Restriction fragment lengt+s +ets start this eer!ise by soving the probem man&ay. 'f e oo- thro&gh the D;4 se<&en!e e !an spot the )!o' site at position 21. 7eres the se<&en!e ith the base positions abee% above an% the )!o' motif in bo%: 1 2 3 ' 0123'"7GJ0123'"7GJ0123'"7GJ0123'"7GJ0123'"7GJ0123' A&)%A)&%A))A&%)A)A%)A GAATTCTA)&A)A&A)A)A)A)&%A)%&%))&A)
"in!e the )!o' enyme !&ts the D;4 beteen the R an% first 4, e !an fig&re o&t that the first fragment i r&n from position 0 to position 21, an% the se!on% fragment from position 22 to the ast position, B. $herefore the engths of the to fragments are 22 an% 33. riting a program to fig&re o&t the engths is 9&st a <&estion of appying the same ogi!. e &se the find metho% to fig&re o&t the position of the start of the )!o' motif, then a%% one to a!!o&nt for the fa!t that the positions start !o&nting from ero 5 this i give &s the ength of the first fragment. From there e !an get the ength of the se!on% fragment by fin%ing the ength of the inp&t se<&en!e an% s&btra!ting the ength of the first fragment: my/dna B *A&)%A)&%A))A&%)A)A%)A%AA))&)A)&A)A&A)A)A)A)&%A)%&%))&A)* frag1/length B my/dna.find(*%AA))&*# C 1 frag2/length B len(my/dna# F frag1/length print(*length of fragment one is * C str(frag1/length## print(*length of fragment to is * C str(frag2/length##
$he o&tp&t from this program !onfirms that it agrees ith the anser e got man&ay: length of fragment one is 22 length of fragment to is 33
BH
Chapter 2: #rinting an% manip&ating tet
'f e ante% to r&n the same program &sing a %ifferent restri!tion enyme, e% have to !hange bot+ the string that e &se% in the find metho% !a, and the n&mber that e a%% in or%er to ta-e a!!o&nt of the !&t site. 'ts orth noting that this program ass&mes that the D;4 se<&en!e %efinitey %oes !ontain the restri!tion site ere oo-ing for. 'f e try the same program &sing a D;4 se<&en!e hi!h %oesnt !ontain the site, it i report a fragment of ength 0 an% a fragment hose ength is e<&a to the tota ength of the D;4 se<&en!e. hie this is not stri!ty rong, its a itte misea%ing 5 if e ere going to &se this program for reaife or-, e% probaby prefer to have sighty %ifferent behavio&r %epen%ing on hether or not the D;4 se<&en!e !ontaine% the motif ere oo-ing for. e ta- abo&t ho to impement that type of behavio&r in !hapter G.
plicing out introns) part one 'n this eer!ise, ere being as-e% to pro%&!e a program that %oes the 9ob of a spi!eosome 5 spits a D;4 se<&en!e at to spe!ifie% o!ations to ma-e three pie!es, then 9oin the o&ter to pie!es together1. +ets start by spitting the se<&en!e &p into three bits. e have to &se the s&bstring notation from earier in the !hapter, an% e nee% to ta-e !are ith the n&mbers. e -no that if e give a stop position for a s&bstring then it i go on to the en% of the inp&t string, so rather than fig&re o&t the position of the en% of the se<&en!e, e 9&st be ay an% &se a big n&mber. 7eres the !o%e the first ine, here e store the D;4 se<&en!e in the variabe my_dna, is too ong to fit on one ine on the page, so it oo-s i-e its sprea% o&t over m&tipe ines=:
1
e -no that thats not reay ho a spi!osome or-s, b&t its fine as a !on!ept&a mo%e.
BI
Chapter 2: #rinting an% manip&ating tet
my/dna B *A)&%A)&%A)&%A)&%A&)%A&)A%)&A)A%&)A)%&A)%)A%&)A&)&%A)&%A)&%A)&%A)&%A)&%A)& %A)&%A)&%A)&A)%&)A)&A)&%A)&%A)A)&%A)%&A)&%A&)A&)A)* exon1 B my/dna$1!"3 exon2 B my/dna$J1!10000 print(exon1 C exon2#
$he o&tp&t from this !o%e oo-s vag&ey right: )&%A)&%A)&%A)&%A&)%A&)A%)&A)A%&)A)%&A)%)A%&)A&)&%A)&%A)&%A)&%A)&A)&%A)&%A) A)&%A)%&A)&%A&)A&)A)
b&t hen e oo- more !osey e !an see that something is not right. $he printe% !o%ing se<&en!e is s&ppose% to start at the very first !hara!ter of the inp&t se<&en!e, b&t its starting at the se!on%. e have forgotten to ta-e into a!!o&nt the fa!t that #ython starts !o&nting from ero, so o&r n&mbers are a too high by one. +ets try again: my/dna B *A)&%A)&%A)&%A)&%A&)%A&)A%)&A)A%&)A)%&A)%)A%&)A&)&%A)&%A)&%A)&%A)&%A)&%A)& %A)&%A)&%A)&A)%&)A)&A)&%A)&%A)A)&%A)%&A)&%A&)A&)A)* exon1 B my/dna$0!"2 exon2 B my/dna$J0!10000 print(exon1 C exon2#
;o the o&tp&t oo-s !orre!t 5 the !o%ing se<&en!e starts at the very beginning of the inp&t se<&en!e: A)&%A)&%A)&%A)&%A&)%A&)A%)&A)A%&)A)%&A)%)A%&)A&)&%A)&%A)&%A)&%A)&A)&%A)&%A )A)&%A)%&A)&%A&)A&)A)
B@
Chapter 2: #rinting an% manip&ating tet
plicing out introns) part to $his is a straightforar% pie!e of n&mber!r&n!hing. $here are a !o&pe of ays to go abo&t it. e !o&% &se the eon startstop !oor%inates to !a!&ate the ength of the !o%ing portion of the se<&en!e. 7oever, sin!e eve area%y ritten the !o%e to generate the !o%ing se<&en!e, e !an simpy !a!&ate the ength of it, an% then %ivi%e by the ength of the inp&t se<&en!e: my/dna B *A)&%A)&%A)&%A)&%A&)%A&)A%)&A)A%&)A)%&A)%)A%&)A&)&%A)&%A)&%A)&%A)&%A)&%A)& %A)&%A)&%A)&A)%&)A)&A)&%A)&%A)A)&%A)%&A)&%A&)A&)A)* exon1 B my/dna$0!"2 exon2 B my/dna$J0!10000 coding/length B len(exon1 C exon2# total/length B len(my/dna# print(coding/length total/length#
$he o&tp&t shos that ere neary right: 0.772377237723G
e have !a!&ate% the !o%ing proportion as a fra!tion, b&t the eer!ise !ae% for a per!entage. e !an easiy fi this by m&tipying by 100. ;oti!e that the symbo for m&tipi!ation is not x, as yo& might thin-, b&t 4. $he fina !o%e: my/dna B *A)&%A)&%A)&%A)&%A&)%A&)A%)&A)A%&)A)%&A)%)A%&)A&)&%A)&%A)&%A)&%A)&%A)&%A)& %A)&%A)&%A)&A)%&)A)&A)&%A)&%A)A)&%A)%&A)&%A&)A&)A)* exon1 B my/dna$0!"2 exon2 B my/dna$J0!10000 coding/length B len(exon1 C exon2# total/length B len(my/dna# print(100 K coding/length total/length#
gives the !orre!t o&tp&t:
0
Chapter 2: #rinting an% manip&ating tet
77.2377237723G
atho&gh e probaby %ont reay re<&ire that n&mber of signifi!ant fig&res. 'n !hapter e i earn ho to format the o&tp&t ni!ey.
plicing out introns) part t+ree $his so&n%s <&ite tri!-y, b&t e have area%y %one the har% bit in part one. 4 e nee% to %o is etra!t the intron se<&en!e as e as the eons, !onvert it to oer !ase, then !on!atenate the three se<&en!es to re!reate the origina genomi! se<&en!e: my/dna B *A)&%A)&%A)&%A)&%A&)%A&)A%)&A)A%&)A)%&A)%)A%&)A&)&%A)&%A)&%A)&%A)&%A)&%A)& %A)&%A)&%A)&A)%&)A)&A)&%A)&%A)A)&%A)%&A)&%A&)A&)A)* exon1 B my/dna$0!"2 intron B my/dna$"2!J0 exon2 B my/dna$J0!10000 print(exon1 C intron.loer(# C exon2#
+oo-ing at the o&tp&t, e see an &pper !ase D;4 se<&en!e ith a oer !ase se!tion in the mi%%e, as epe!te%: A)&%A)&%A)&%A)&%A&)%A&)A%)&A)A%&)A)%&A)%)A%&)A&)&%A)&%A)&%A)&%atcgatcgatcg atcgatcgatcatgctA)&A)&%A)&%A)A)&%A)%&A)&%A&)A&)A)
hen e are appying severa transformations to tet, as in this eer!ise, there are &s&ay a n&mber of %ifferent ays e !an rite the program. For eampe, e !o&% store the oer !ase version of the intron, rather than !onverting it to oer !ase hen printing: intron B my/dna$"2!J0.loer(#
1
Chapter 2: #rinting an% manip&ating tet
?r e !o&% avoi% &sing variabes for the introns an% eons a together, an% %o everything in one big print statement: print(my/dna$0!"2 C my/dna$"2!J0.loer(# C my/ dna$J0!10000#
$his ast option is very !on!ise, b&t a bit har%er to rea% than the more verbose ay. 4s the eer!ises in this boo- get onger, yo& noti!e that there are more an% more %ifferent ays to rite the !o%e 5 yo& may en% &p ith so&tions that oo- very %ifferent to the eampe so&tions. hen trying to !hoose beteen %ifferent ays to rite a program, aays favo&r the so&tion that is !earest in intent an% easiest to rea%.
2
Chapter 3: ea%ing an% riting fies
3:
Reading and riting files
Why are we so intereste" in working with files? 4s e start this !hapter, e fin% o&rseves on!e again %oing things in a sighty %ifferent or%er to most programming boo-s. $he ma9ority of intro%&!tory programming boo-s ont !onsi%er or-ing ith eterna fies &nti m&!h f&rther aong, so hy are e intro%&!ing it noE $he anser, as as the !ase in the ast !hapter, ies in the parti!&ar 9obs that e ant to &se #ython for. $he %ata that e as bioogists or- ith is store% in fies, so if ere going to rite &sef& programs e nee% a ay to get the %ata o&t of fies an% into o&r programs an% vice versa=. 4s yo& ere going thro&gh the eer!ises in the previo&s !hapter, it may have o!!&rre% to yo& that !opying an% pasting a D;4 se<&en!e %ire!ty into a program ea!h time e ant to &se it is not a very goo% approa!h to ta-e, an% yo&% be right. $he se<&en!es e ere or-ing ith in the eer!ises ere very short obvio&sy reaife %ata i be m&!h onger. 4so, it seems ineegant to have the %ata e ant to or- on mie% &p ith the !o%e that manip&ates it. 'n this !hapter e see a better ay to %o it. ere &!-y in bioogy that many of the types of %ata that e or- ith are store% in tet1 fies hi!h are reativey simpe to pro!ess &sing #ython. Chief among these, of !o&rse, are D;4 an% protein se<&en!e %ata, hi!h !an be store% in a variety of formats.2 (&t there are many other types of %ata 5 se<&en!ing rea%s, <&aity s!ores, ";#s, phyogeneti! trees, rea% maps, geographi!a sampe %ata, geneti! %istan!e matri!es 5 hi!h e !an a!!ess from ithin o&r #ython programs.
1 2
i.e. fies hi!h yo& !an open in a tet e%itor an% rea%, as oppose% to binary fies hi!h !annot be rea% %ire!ty. 'n this boo- e mosty be ta-ing abo&t F4"$4 format as its the simpest an% most !ommon format, b&t there are many more.
3
Chapter 3: ea%ing an% riting fies
4nother reason for o&r interest in fie inp&t/o&tp&t is the nee% for o&r #ython programs to or- as part of a pipeine or or- fo invoving other, eisting toos. hen it !omes to &sing #ython in the rea or%, e often ant #ython to either a!!ept %ata from, or provi%e %ata to, another program. ?ften the easiest ay to %o this is to have #ython rea%, or rite, fies in a format that the other program area%y &n%erstan%s.
(ea"ing te!t from a file Firsty, a <&i!- note abo&t hat e mean by tet. 'n programming, hen e taabo&t te!t files, e are not ne!essariy ta-ing abo&t something that is h&man rea%abe. ather, e are ta-ing abo&t a fie that !ontains !hara!ters an% ines 5 something that yo& !o&% open &p an% vie in a tet e%itor, regar%ess of hether yo& !o&% a!t&ay ma-e sense of the fie or not. )ampes of tet fies hi!h yo& might have en!o&ntere% in!&%e: •
F4"$4 fies of D;4 or protein se<&en!es
•
fies !ontaining o&tp&t from !omman%ine programs e.g. (+4"$=
•
F4"$L fies !ontaining D;4 se<&en!ing rea%s
•
7$M+ fies
•
or% pro!essing %o!&ments
•
an% of !o&rse, #ython !o%e
'n !ontrast, most fies that yo& en!o&nter %ayto%ay i be binary files 5 ones hi!h are not ma%e &p of !hara!ters an% ines, b&t of bytes. )ampes in!&%e: •
image fies J#)Rs an% #;Rs=
•
a&%io fies
•
vi%eo fies
B
Chapter 3: ea%ing an% riting fies •
!ompresse% fies e.g. X'# fies=
'f yo&re not s&re hether a parti!&ar fie is tet or binary, theres a very simpe ay to te 5 9&st open it &p in a tet e%itor. 'f the fie %ispays itho&t any probem, then its tet regar%ess of hether yo& !an ma-e sense of it or not=. 'f yo& get an error or a arning from yo&r tet e%itor, or the fie %ispays as a !oe!tion of in%e!ipherabe !hara!ters, then its binary. $he eampes an% eer!ises in this !hapter are a itte %ifferent from those in the previo&s one, be!a&se they rey on the eisten!e of the fies that e are going to manip&ate. 'f yo& ant to try r&nning the eampes in this !hapter, yo& nee% to ma-e s&re that there is a fie in yo&r or-ing %ire!tory !ae% "na=t!t hi!h has a singe ine !ontaining a D;4 se<&en!e. $he easiest ay to %o this is to r&n the eampes hie in the chapter) fo%er insi%e the eer!ises %onoa%1.
5sing open to read a file 'n #ython, as in the physi!a or%, e have to open a fie before e !an rea% hats insi%e it. $he #ython f&n!tion that !arries o&t the 9ob of opening a fie is very sensiby !ae% open. 't ta-es one arg&ment 5 a string hi!h !ontains the name of the fie 5 an% ret&rns a file ob@ect my/file B open(*dna.txt*#
4 file ob@ect is a ne type hi!h e havent en!o&ntere% before, an% its a itte more !ompi!ate% than the string an% n&mber types that e sa in the previo&s !hapter. ith strings an% n&mbers it as easy to &n%erstan% hat they represente% 5 a singe bit of tet, or a singe n&mber. 4 fie ob9e!t, in !ontrast, represents something a bit ess tangibe 5 it represents a fie on yo&r !omp&ters har% %rive.
1
'f yo& havent %onoa%e% the eampe fies yet, yo& fin% the in- in the emai that !ame ith yo&r p&r!hase of this boo-.
Chapter 3: ea%ing an% riting fies
$he ay that e &se fie ob9e!ts is a bit %ifferent to strings an% n&mbers as e. 'f yo& gan!e ba!- at the eampes from the previo&s !hapter yo& see that most of the time hen e ant to &se a variabe !ontaining a string or n&mber e 9&st &se the variabe name: my/string B a-cdefg print(my/string# my/num-er B '2 print(my/num-er C 1#
'n !ontrast, hen ere or-ing ith fie ob9e!ts most of o&r intera!tion i be thro&gh metho"s. $his stye of programming i seen &n&s&a at first, b&t as e see in this !hapter, the fie type has a e tho&ghto&t set of metho%s hi!h et &s %o ots of &sef& things. $he first thing e nee% to be abe to %o is to rea% the !ontents of the fie. $he fie type has a read metho% hi!h %oes this. 't %oesnt ta-e any arg&ments, an% the ret&rn va&e is a string, hi!h e !an store in a variabe. ?n!e eve rea% the fie !ontents into a variabe, e !an treat them 9&st i-e any other string 5 for eampe, e !an print them: my/file B open(*dna.txt*# file/contents B my/file.read(# print(file/contents#
/iles0 contents an" file names hen earning to or- ith fies its very easy to get !onf&se% beteen a file ob@ect , a file name , an% the contents of a fie. $a-e a oo- at the fooing bit of !o%e:
G
Chapter 3: ea%ing an% riting fies
1
my/file/name B *dna.txt* 2 my/file B open(my/file/name# 3 my/file/contents B my/file.read(#
hats going on hereE ?n ine 1, e store the string "na=t!t in the variabe my_file_name . ?n ine 2, e &se the variabe my_file_name as the arg&ment to the open f&n!tion, an% store the res&ting fie ob9e!t in the variabe my_file. ?n ine 3, e !a the read metho% on the variabe my_file, an% store the res&ting string in the variabe my_file_contents . $he important thing to &n%erstan% abo&t this !o%e is that there are three separate variabes hi!h have %ifferent types an% hi!h are storing three very %ifferent things. my_file_name is a string, an% it stores the name of a fie on %is-. my_file is a fie ob9e!t, an% it represents the fie itsef. my_file_contents is a string, an% it stores the tet that is in the fie. emember that variabe names are arbitrary 5 the !omp&ter %oesnt !are hat yo& !a yo&r variabes. "o this pie!e of !o%e is ea!ty the same as the previo&s eampe: apple B *dna.txt* -anana B open(apple# grape B -anana.read(#
e!ept it is har%er to rea%8 'n !ontrast, the fie name "na=t!t = is not arbitrary 5 it m&st !orrespon% to the name of a fie on the har% %rive of yo&r !omp&ter. 4 !ommon error is to try to &se the read metho% on the rong thing. e!a that read is a metho% that ony or-s on fie ob9e!ts. 'f e try to &se the read metho% on the fie name: my/file/name B *dna.txt* my/contents B my/file/name.read(#
H
Chapter 3: ea%ing an% riting fies
e get an !ttri$uteError 5 #ython i !ompain that strings %ont have a read metho%1: Attri-ute9rror! str o-Dect has no attri-ute read
4nother !ommon error is to &se the file ob@ect hen e meant to &se the file contents. 'f e try to print the fie ob9e!t: my/file/name B *dna.txt* my/file B open(my/file/name# print(my/file#
e ont get an error, b&t e get an o%%oo-ing ine of o&tp&t: ;open file dna.txt, mode r at 0x7fcff77G'-0+
e ont %is!&ss the meaning of this ine no: 9&st remember that if yo& try to print the !ontents of a fie b&t instea% yo& get some o&tp&t that oo-s i-e the above, yo& have amost %efinitey printe% the fie ob9e!t rather than the fie !ontents.
ealing with newlines +ets ta-e a oo- at the o&tp&t e get hen e try to print some information from a fie. e &se the "na=t!t fie from the chapter) eer!ises fo%er. $his fie !ontains a singe ine ith a short D;4 se<&en!e. ?pen the fie &p in a tet e%itor an% ta-e a oo- at it. ere going to rite a simpe program to rea% the D;4 se<&en!e from the fie an% print it o&t aong ith its ength. #&tting together the fie f&n!tions an% metho%s
1
From no on, ' 9&st sho the reevant bits of o&tp&t hen %is!&ssing error message.
I
Chapter 3: ea%ing an% riting fies
from this !hapter, an% the materia e sa in the previo&s !hapter, e get the fooing !o%e: 4 open the file my/file B open(*dna.txt*# 4 read the contents my/dna B my/file.read(# 4 calculate the length dna/length B len(my/dna# 4 print the output print(*seuence is * C my/dna C
* and length is * C str(dna/length##
hen e oo- at the o&tp&t, e !an see that the program is or-ing amost perfe!ty 5 b&t there is something strange: the o&tp&t has been spit over to ines: seuence is A&)%)A&%)%&A&)%A)& and length is 1J
$he epanation is simpe on!e yo& -no it: #ython has in!&%e% the ne ine !hara!ter at the en% of the "na=t!t fie as part of the !ontents. 'n other or%s, the variabe my_dna has a ne ine !hara!ter at the en% of it. 'f e !o&% vie the my_dna variabe %ire!ty 1, e o&% see that it oo-s i-e this: A&)%)A&%)%&A&)%A)&n
$he so&tion is aso simpe. (e!a&se this is s&!h a !ommon probem, strings have a metho% for removing ne ines from the en% of them. $he metho% is !ae% rstrip, an% it ta-es one string arg&ment hi!h is the !hara!ter that yo& ant to remove. 'n this !ase, e ant to remove the neine !hara!ter \n=. 7eres a mo%ifie% version of the !o%e 5 note that the arg&ment to rstrip is itsef a string so nee%s to be en!ose% in <&otes:
1
'n fa!t, e !an %o this 5 theres a f&n!tion !ae% repr that ret&rns a representation of a variabe.
@
Chapter 3: ea%ing an% riting fies
my/file B open(*dna.txt*# my/file/contents B my/file.read(# 4 remoe the neline from the end of the file contents my/dna B my/file/contents.rstrip(*n*# dna/length B len(my/dna# print(*seuence is * C my/dna C * and length is * C str(dna/length##
an% no the o&tp&t oo-s 9&st as e epe!te%: seuence is A&)%)A&%)%&A&)%A)& and length is 1G
'n the !o%e above, e first rea% the fie !ontents an% then remove% the neine, in to separate steps: my/file/contents B my/file.read(# my/dna B my/file/contents.rstrip(*n*#
b&t its more !ommon to rea% the !ontents an% remove the neine a in one go, i-e this: my/dna B my/file.read(#.rstrip(*n*#
$his is a bit tri!-y to rea% at first as e are &sing to %ifferent metho%s read an% rstrip= in the same statement. $he -ey is to rea% it from eft to right 5 e ta-e the my_file variabe an% &se the read metho% on it, then e ta-e the o&tp&t of that metho% hi!h e -no is a string= an% &se the rstrip metho% on it. $he res&t of the rstrip metho% is then store% in the my_dna variabe. 'f yo& fin% it %iffi!&t rite the hoe thing as one statement i-e this, 9&st brea- it &p an% %o the to things separatey 5 yo&r programs i r&n 9&st as e.
G0
Chapter 3: ea%ing an% riting fies
issing files hat happens if e try to rea% a fie that %oesnt eistE my/file B open(*nonexistent.txt*#
e get a ne type of error that eve not seen before: L>9rror! $9rrno 2
'%eay, e% i-e to be abe to !he!- if a fie eists before e try to open it 5 e see ho to %o that in !hapter @.
Writing te!t to files 4 the eampe programs that eve seen so far in this boo- have pro%&!e% o&tp&t straight to the s!reen. $hats great for eporing ne feat&res an% hen or-ing on programs, be!a&se it aos yo& to see the effe!t of !hanges to the !o%e right aay. 't has a fe %raba!-s, hoever, hen riting !o%e that e might ant to &se in rea ife. #rinting o&tp&t to the s!reen ony reay or-s e hen there isnt very m&!h of it1. 'ts great for short programs an% stat&s messages, b&t <&i!-y be!omes !&mbersome for arge amo&nts of o&tp&t. "ome terminas str&gge ith arge amo&nts of tet, or orse, have a imite% s!roba!- !apabiity hi!h !an !a&se the first bit of yo&r o&tp&t to %isappear. 'ts not easy to sear!h in o&tp&t thats being %ispaye% at the termina, an% ong ines ten% to get rappe%. 4so, for many programs e ant to sen% %ifferent bits of o&tp&t to %ifferent fies, rather than having it a %&mpe% in the same pa!e.
1
+in& &sers may be aare that e !an re%ire!t termina o&tp&t to a fie &sing she re%ire!tion, hi!h !an get aro&n% some of these probems.
G1
Chapter 3: ea%ing an% riting fies
Most importanty, termina o&tp&t vanishes hen yo& !ose yo&r termina program. For sma programs i-e the eampes in this boo-, thats not a probem 5 if yo& ant to see the o&tp&t again yo& !an 9&st rer&n the program. 'f yo& have a program that re<&ires a fe ho&rs to r&n, thats not s&!h a great option.
6pening files for riting 'n the previo&s se!tion, e sa ho to open a fie an% rea% its !ontents. e !an aso open a fie an% write some %ata to it, b&t e have to &se the open f&n!tion in a sighty %ifferent ay. $o open a fie for riting, e &se a toarg&ment version of the open f&n!tion, here the se!on% arg&ment is a spe!iayformatte% string %es!ribing hat e ant to %o to the fie 1. $his se!on% arg&ment !an be >r> for rea%ing, >> for riting, or >a> for appen%ing 2. 'f e eave o&t the se!on% arg&ment i-e e %i% for a the eampes above=, #ython &ses the %efa&t, hi!h is >r> for rea%ing. $he %ifferen!e beteen >> an% >a> is s&bte, b&t important. 'f e open a fie that area%y eists &sing the mo%e >>, then e i overrite the !&rrent !ontents ith hatever %ata e rite to it. 'f e open an eisting fie ith the mo%e >a>, it i a%% ne %ata onto the en% of the fie, b&t i not remove any eisting !ontent. 'f there %oesnt area%y eist a fie ith the spe!ifie% name, then >> an% >a> behave i%enti!ay 5 they i both !reate a ne fie to ho% the o&tp&t. L&ite a ot of #ython f&n!tions an% metho%s have these optiona arg&ments. For the p&rposes of this boo-, e i ony mention them hen they are %ire!ty reevant to hat ere %oing. 'f yo& ant to see a the optiona arg&ments for a parti!&ar metho% or f&n!tion, the best pa!e to oo- is the offi!ia #ython %o!&mentation 5 see !hapter 1 for %etais.
1 2
e !a this the mo"e of the fie. $hese are the most !ommony&se% options 5 there are a fe others.
G2
Chapter 3: ea%ing an% riting fies
?n!e eve opene% a fie for riting, e !an &se the fie rite metho% to rite some tet to it. rite or-s a ot i-e print 5 it ta-es a singe string arg&ment b&t instea% of printing the string to the s!reen it rites it to the fie. 7eres ho e &se open ith a se!on% arg&ment to open a fie an% rite a singe ine of tet to it: my/file B open(*out.txt*, **# my/file.rite(*ello orld*#
(e!a&se the o&tp&t is being ritten to the fie in this eampe, yo& ont see any o&tp&t on the s!reen if yo& r&n it. $o !he!- that the !o%e has or-e%, yo& have to r&n it, then open &p the fie out=t!t in yo&r tet e%itor an% !he!- that its !ontents are hat yo& epe!t 1. emember that ith rite, 9&st i-e ith print, e !an &se an* string as the arg&ment. $his aso means that e !an &se any metho% or f&n!tion that returns a string. $he fooing are a perfe!ty ?K: 4 rite *a-cdef* my/file.rite(*a-c* C *def*# 4 rite *G* my/file.rite(str(len(A%)%&)A%### 4 rite *))%&* my/file.rite(*A)%&*.replace(A, )## 4 rite *atgc* my/file.rite(*A)%&*.loer(## 4 rite contents of my/aria-le my/file.rite(my/aria-le#
1
.tt is the stan%ar% fie name etension for a pain tet fie. +ater in this boo-, hen e generate o&tp&t fies ith a parti!&ar format, e &se %ifferent fie name etensions.
G3
Chapter 3: ea%ing an% riting fies
4losing files $heres one more important fie metho% to oo- at before e finish this !hapter 5 close. *ns&rprisingy, this is the opposite of open b&t note that its a metho" , hereas open is a function=. e sho&% !a close after ere %one rea%ing or riting to a fie 5 e ont go into the %etais here, b&t its a goo% habit to get into as it avoi%s some types of b&gs that !an be tri!-y to tra!- %on 1. close is an &n&s&a metho% as it ta-es no arg&ments so its !ae% ith an empty pair of parentheses= an% %oesnt ret&rn any &sef& va&e: my/file B open(*out.txt*, **# my/file.rite(*ello orld*# 4 remem-er to close the file my/file.close(#
Paths an" fol"ers "o far, e have ony %eat ith opening fies in the same fo%er that e are r&nning o&r program. hat if e ant to open a fie from a %ifferent part of the fie systemE $he open f&n!tion is <&ite happy to %ea ith fies from anyhere on yo&r !omp&ter, as ong as yo& give the f& path i.e. the se<&en!e of fo%er names that tes yo& the o!ation of the fie=. J&st give a file path as the arg&ment rather than a file name. $he format of the fie path oo-s %ifferent %epen%ing on yo&r operating system. 'f yo&re on +in&, it i oo- i-e this: my/file B open(*homemartinmyfoldermyfile.txt*#
if yo&re on in%os, i-e this 2: 1 2
"pe!ifi!ay, it heps to ens&re that o&tp&t to a fie is f&she%, hi!h is ne!essary hen e ant to ma-e a fie avaiabe to another program as part of o&r or- fo. $he etra r !hara!ter before the string is ne!essary to prevent #ython from trying to interpret the ba!-sash in the fie path see !hapter H for an epanation.
GB
Chapter 3: ea%ing an% riting fies
my/file B open(r*c!indos@es:topmyfoldermyfile.txt*#
an% if yo&re on a Ma!, i-e this: my/file B open(*Msersmartin@es:topmyfoldermyfile.txt*#
(ecap eve ta-en a hoe !hapter to intro%&!e the vario&s ays of rea%ing an% riting to fies, be!a&se its s&!h an important part of b&i%ing programs that are &sef& in bioogy. eve seen ho or-ing ith fie !ontents is aays a tostep pro!ess 5 e m&st open a fie before rea%ing or riting 5 an% oo-e% at severa !ommon pitfas. e ret&rn to the theme of fie manip&ation in ater !hapters here e a%%ress some of the short!omings of the te!hni<&es e earne% in this !hapter: •
4 the eampes in this !hapter than invove rea%ing fies %o so by rea%ing a the !ontent into a singe variabe. ?ften, this is not hat e ant 5 a m&!h more !ommon re<&irement is to pro!ess a fie inebyine. 'n !hapter B e earn abo&t ists an% oops, hi!h i ao &s to %o ea!ty that.
•
$here are, of !o&rse, many things e ant to %o to fies besi%es simpy rea%ing their !ontents. e o&% aso i-e o&r programs to be abe to move an% !opy fies, to !reate an% %eete fies an% %ire!tories, an% to ist fies an% their properties. e !over the toos re<&ire% to %o these things in !hapter @.
•
Finay, another feat&re !ommon to a o&r eampes is that the names of fies are ritten as part of the !o%e. e i generay ant o&r reaife programs to be more feibe, an% !apabe of rea%ing fies that are spe!ifie% by the &ser. Chapter @ aso %eas ith the vario&s forms of &ser inp&t an% in it e earn ho to ma-e o&r programs a!!ept fie names feiby.
G
Chapter 3: ea%ing an% riting fies
!ercises plitting genomic '.4 +oo- in the chapter) fo%er for a fie !ae% genomic"na=t!t 5 it !ontains the same pie!e of genomi! D;4 that e ere &sing in the fina eer!ise from !hapter 2. rite a program that i spit the genomi! D;4 into !o%ing an% non!o%ing parts, an% rite these se<&en!es to to separate fies. 7int: &se yo&r so&tion to the ast eer!ise from !hapter 2 as a starting point.
%riting a (4T4 file F4"$4 fie format is a !ommony&se% D;4 an% protein se<&en!e fie format. 4 singe se<&en!e in F4"$4 format oo-s i-e this: +seuence/name A)&%A&)%A)&%A)&%)A&%A)
here se<&en!eYname is a hea%er that %es!ribes the se<&en!e the greaterthan symbo in%i!ates the start of the hea%er ine=. ?ften, the hea%er !ontains an a!!ession n&mber that reates to the re!or% for the se<&en!e in a p&bi! se<&en!e %atabase. 4 singe F4"$4 fie !an !ontain m&tipe se<&en!es, i-e this: +seuence/one A)&%A)&%A)&%A)&%A) +seuence/to A&)A%&)A%&)A%&A)&% +seuence/three A&)%&A)&%A)&%)A&&)
GG
Chapter 3: ea%ing an% riting fies
rite a program that i !reate a F4"$4 fie for the fooing three se<&en!es 5 ma-e s&re that a se<&en!es are in &pper !ase an% ony !ontain the bases 4, $, R an% C. euence +eader
'.4 seuence
!5C36/
!"C#"!C#!"C#!"C#!"C#C"!#!C#"!"C#
(E7819
actgatcgacgatcgatcgatcacgact
:;<=>?
!C"#!C-!C"#"--!C"#"!----C!"#"#
%riting multiple (4T4 files *se the %ata from the previo&s eer!ise, b&t instea% of !reating a singe F4"$4 fie, !reate three ne F4"$4 fies 5 one per se<&en!e. $he names of the F4"$4 fies sho&% be the same as the se<&en!e hea%er names, ith the etension =fasta.
GH
Chapter 3: ea%ing an% riting fies
&olutions plitting genomic '.4 e have a hea%start on this probem, be!a&se e have area%y ta!-e% a simiar probem in the previo&s !hapter. +ets remin% o&rseves of the so&tion e en%e% &p ith for that eer!ise: my/dna B *A)&%A)&%A)&%A)&%A&)%A&)A%)&A)A%&)A)%&A)%)A%&)A&)&%A)&%A)&%A)&%A)&%A)&%A)& %A)&%A)&%A)&A)%&)A)&A)&%A)&%A)A)&%A)%&A)&%A&)A&)A)* exon1 B my/dna$0!"2 intron B my/dna$"2!J0 exon2 B my/dna$J0!10000 print(exon1 C intron.loer(# C exon2#
hat !hanges %o e nee% to ma-eE Firsty, e nee% to rea% the D;4 se<&en!e from a fie instea% of riting it in the !o%e: dna/file B open(*genomic/dna.txt*# my/dna B dna/file.read(#
"e!on%y, e nee% to !reate to ne fie ob9e!ts to ho% the o&tp&t: coding/file B open(*coding/dna.txt*, **# noncoding/file B open(*noncoding/dna.txt*, **#
Finay, e nee% to !on!atenate the to eon se<&en!es an% rite them to the !o%ing D;4 fie, an% rite the intron se<&en!e to the non!o%ing D;4 fie: coding/file.rite(exon1 C exon2# noncoding/file.rite(intron#
GI
Chapter 3: ea%ing an% riting fies
+ets p&t it a together, ith some ban- ines to separate o&t the %ifferent parts of the program: 4 open the file and read its contents dna/file B open(*genomic/dna.txt*# my/dna B dna/file.read(# 4 extract the different -its of @
%riting a (4T4 file +ets start this probem by thin-ing abo&t the variabes ere going to nee%. e have three D;4 se<&en!es in tota, so e nee% three variabes to ho% the se<&en!e hea%ers, an% three more to ho% the se<&en!es themseves: header/1 B *!5C36/ header_6 @ (E7819 header_/ @ :;<=>? seA_3 @ !"C#"!C#!"C#!"C#!"C#C"!#!C#"!"C# seA_6 @ actgatcgacgatcgatcgatcacgact seA_/ @ !C"#!C-!C"#"--!C"#"!----C!"#"#
F4"$4 format has aternating ines of hea%er an% se<&en!e, so before e try any se<&en!e manip&ation, ets try to rite a program that pro%&!es the ines in the right or%er. ather than riting to a fie, e print the o&tp&t to the s!reen for no
G@
Chapter 3: ea%ing an% riting fies
5 that i ma-e it easier to see the o&tp&t right aay. ?n!e eve got it or-ing, e sit!h over to fie o&tp&t. 7eres a fe ines hi!h i print %ata to the s!reen: print(header/1# print(se/1# print(header/2# print(se/2# print(header/3# print(se/3#
an% heres hat the o&tp&t oo-s i-e: AN&123 A)&%)A&%A)&%A)&%A)&%&)A%A&%)A)&% @9'" actgatcgacgatcgatcgatcacgact @9'" actgatcgacgatcgatcgatcacgact
;ot far off 5 the ines are in the right or%er, b&t e forgot to in!&%e the greater than symbo at the start of the hea%er. 4so, e %ont reay nee% to print the hea%er an% the se<&en!e separatey for ea!h se<&en!e 5 e !an in!&%e a neine !hara!ter in the print string in or%er to get them on separate ines. 7eres an improve% version of the !o%e: print(+ C header/1 C n C se/1# print(+ C header/2 C n C se/2# print(+ C header/3 C n C se/3#
an% the o&tp&t oo-s better too:
H0
Chapter 3: ea%ing an% riting fies
+AN&123 A)&%)A&%A)&%A)&%A)&%&)A%A&%)A)&% +@9'" actgatcgacgatcgatcgatcacgact +LH7GJ A&)%A&FA&)%)FFA&)%)AFFFF&A)%)%
;et, ets ta!-e the probems ith the se<&en!es. $he se!on% se<&en!e is in oer !ase, an% it nee%s to be in &pper !ase 5 e !an fi that &sing the upper string metho%. $he thir% se<&en!e has a b&n!h of gaps that e nee% to remove. e havent !ome a!ross a remove metho%.... b&t e %o -no ho to repa!e one !hara!ter ith another. 'f e repa!e a the gap !hara!ters ith an empty string, it i be the same as removing them1. 7eres a version that fies both se<&en!es: print(+ C header/1 C n C se/1# print(+ C header/2 C n C se/2.upper(## print(+ C header/3 C n C se/3.replace(F, ##
;o the printe% o&tp&t is perfe!t: +AN&123 A)&%)A&%A)&%A)&%A)&%&)A%A&%)A)&% +@9'" A&)%A)&%A&%A)&%A)&%A)&A&%A&) +LH7GJ A&)%A&A&)%)A&)%)A&A)%)%
$he fina step is to sit!h from printe% o&tp&t to riting to a fie. e open a ne fie, an% !hange the three print ines to rite:
1
4n empty string is 9&st a pair of open an% !ose <&otation mar-s ith nothing in beteen them.
H1
Chapter 3: ea%ing an% riting fies
output B open(*seuences.fasta*, **# output.rite(+ C header/1 C n C se/1# output.rite(+ C header/2 C n C se/2.upper(## output.rite(+ C header/3 C n C se/3.replace(F, ##
4fter ma-ing these !hanges the !o%e %oesnt pro%&!e any o&tp&t on the s!reen, so to see hats happene% e nee% to ta-e a oo- at the se;uences=fasta fie: +AN&123 A)&%)A&%A)&%A)&%A)&%&)A%A&%)A)&%+@9'" A&)%A)&%A&%A)&%A)&%A)&A&%A&)+LH7GJ A&)%A&A&)%)A&)%)A&A)%)%
$his %oesnt oo- right 5 the se!on% an% thir% ines have been 9oine% together, as have the fo&rth an% fifth. hat has happene%E 't oo-s i-e eve &n!overe% a %ifferen!e beteen the print f&n!tion an% the rite metho%. print a&tomati!ay p&ts a ne ine at the en% of the string, hereas rite %oesnt. $his means eve got to be !aref& hen sit!hing beteen them8 $he fi is <&ite simpe, e 9&st a%% a neine onto the en% of ea!h string that gets ritten to the fie: output B open(*seuences.fasta*, **# output.rite(+ C header/1 C n C se/1 C n# output.rite(+ C header/2 C n C se/2.upper(# C n# output.rite(+ C header/3 C n C se/3.replace(F, # C n#
$he arg&ments for the rite statements are getting <&ite !ompi!ate%, b&t they are a ma%e &p of simpe b&i%ing bo!-s. For eampe the ast one, if e transate% it into )ngish, o&% rea% > a greaterBthan symbol0 followe" by the variable hea"er)0 followe" by a newline0 followe" by the variable se;) with all hyphens replace" with nothing0 followe" by another newline>.
H2
Chapter 3: ea%ing an% riting fies
7eres the fina !o%e, in!&%ing the variabe %efinition at the beginning, ith banines an% !omments: 4 set the alues of all the header aria-les header/1 B *!5C36/ header_6 @ (E7819 header_/ @ :;<=>? B set seA_3 seA_6 seA_/
the alues of all the seAuence aria$les @ !"C#"!C#!"C#!"C#!"C#C"!#!C#"!"C# @ actgatcgacgatcgatcgatcacgact @ !C"#!C-!C"#"D!C"#"!----C!"#"#
B make a ne file to hold the output output @ open+seAuences.fasta% , B rite the header and seAuence for seA3 output.rite+)) F header_3 F )\n) F seA_3 F )\n), B rite the header and uppercase seAuences for seA6 output.rite+)) F header_6 F )\n) F seA_6.upper+, F )\n), B rite the header and seAuence for seA/ ith hyphens remoed output.rite+)) F header_/ F )\n) F seA_/.replace+)-)% )), F )\n),
%riting multiple (4T4 files e !an sove this probem ith a sight mo%ifi!ation of o&r so&tion to the previo&s eer!ise. e nee% to !reate three ne fies to ho% the o&tp&t, an% e !onstr&!t the name of ea!h fie by &sing string !on!atenation: output/1 B open(header/1 C *.fasta*, **# output/2 B open(header/2 C *.fasta*, **# output/3 B open(header/3 C *.fasta*, **#
emember, the first arg&ment to open is a string, so its fine to &se a !on!atenation be!a&se e -no that the res&t of !on!atenating to strings is aso a string.
H3
Chapter 3: ea%ing an% riting fies
e aso !hange the rite statements so that e have one for ea!h of the o&tp&t fies. e nee% to be !aref& ith the n&mber here in or%er to ma-e s&re that e get the right se<&en!e in ea!h fie. 7eres the fina !o%e, ith !omments. 4 set the alues of all the header aria-les header/1 B *AN&123* header/2 B *@9'"* header/3 B *LH7GJ* 4 set se/1 se/2 se/3
the alues of all the seuence aria-les B *A)&%)A&%A)&%A)&%A)&%&)A%A&%)A)&%* B *actgatcgacgatcgatcgatcacgact* B *A&)%A&FA&)%)OA&)%)AFFFF&A)%)%*
4 ma:e three files to hold output/1 B open(header/1 C output/2 B open(header/2 C output/3 B open(header/3 C 4 rite one seuence output/1.rite(+ C output/2.rite(+ C output/3.rite(+ C
the output *.fasta*, **# *.fasta*, **# *.fasta*, **#
to each output file header/1 C n C se/1 C n# header/2 C n C se/2.upper(# C n# header/3 C n C se/3.replace(F, # C n#
+oo-ing at the !o%e above, it seems i-e theres a ot of re%&n%an!y there. )a!h of the fo&r se!tions of !o%e 5 setting the hea%er va&es, setting the se<&en!e va&es, !reating the o&tp&t fies, an% riting %ata to the o&tp&t fies 5 !onsists of three nearyi%enti!a statements. 4tho&gh the so&tion or-s, it seems to invove a ot of &nne!essary typing8 4so, having so m&!h nearyi%enti!a !o%e seems i-ey to !a&se errors if e nee% to !hange something. 'n the net !hapter, e eamine some toos hi!h i ao &s to start removing some of that re%&n%an!y.
HB
Chapter B: +ists an% oops
":
#ists and loops
Why "o we nee" lists an" loops? $hin- ba!- over the eer!ises that eve seen in the previo&s to !hapters 5 theyve a invove% %eaing ith one bit of information at a time. 'n !hapter 2, e &se% string manip&ation toos to pro!ess singe se<&en!es, an% in !hapter 3, e pra!tise% rea%ing an% riting fies one at a time. $he !osest e got to &sing m&tipe pie!es of %ata as %&ring the fina eer!ise in !hapter 3, here e ere %eaing ith three D;4 se<&en!es. 'f thats a that #ython aoe% &s to %o, it o&%nt be a very hepf& too for bioogy. 'n fa!t, theres a goo% !han!e that yo&re rea%ing this boo- be!a&se yo& ant to be abe to rite programs to hep yo& %ea ith arge %atasets. 4 very !ommon sit&ation in bioogi!a resear!h is to have a arge !oe!tion of %ata D;4 se<&en!es, ";# positions, gene epression meas&rements= that a nee% to be pro!esse% in the same ay. 'n this !hapter, e earn abo&t the f&n%amenta programming toos that i ao o&r programs to %o this. "o far e have earne% abo&t severa %ifferent %ata types strings, n&mbers, an% fie ob9e!ts=, a of hi!h store a singe bit of information 1. hen eve nee%e% to store m&tipe bits of information for eampe, the three D;4 se<&en!es in the !hapter 3 eer!ises= e have simpy !reate% more variabes to ho% them: 4 set se/1 se/2 se/3
1
the alues of all the seuence aria-les B *A)&%)A&%A)&%A)&%A)&%&)A%A&%)A)&%* B *actgatcgacgatcgatcgatcacgact* B *A&)%A&FA&)%)OA&)%)AFFFF&A)%)%*
e -no that fies are sighty %ifferent to strings an% n&mbers be!a&se they !an store a ot of information, b&t ea!h fie ob9e!t sti ony refers to a singe fie.
H
Chapter B: +ists an% oops
$he imitations of this approa!h be!ame !ear <&ite <&i!-y as e oo-e% at the so&tion !o%e 5 it ony or-e% be!a&se the n&mber of se<&en!es ere sma, an% e -ne the n&mber in a%van!e. 'f e ere to repeat the eer!ise ith three h&n%re% or three tho&san% se<&en!es, the vast ma9ority of the !o%e o&% be given over to storing variabes an% it o&% be!ome !ompetey &nmanageabe. 4n% if e ere to try an% rite a program that !o&% pro!ess an &n-non n&mber of inp&t se<&en!es for instan!e, by rea%ing them from a fie=, e o&%nt be abe to %o it. $o ma-e o&r programs abe to pro!ess m&tipe pie!es of %ata, e nee% an entirey ne type of str&!t&re hi!h !an ho% many pie!es of information at the same time 5 a list . eve aso %eat e!&sivey ith programs hose statements are ee!&te% from top to bottom in a very straightforar% ay. $his has great a%vantages hen first starting to thin- abo&t programming 5 it ma-es it very easy to foo the fo of a program. $he %onsi%e of this se<&entia stye of programming, hoever, is that it ea%s to very re%&n%ant !o%e i-e e sa at the en% of the previo&s !hapter: 4 ma:e three files to hold output/1 B open(header/1 C output/2 B open(header/2 C output/3 B open(header/3 C
the output *.fasta*, **# *.fasta*, **# *.fasta*, **#
4gain it as ony possibe to sove the eer!ise in this manner be!a&se e -ne in a%van!e the n&mber of o&tp&t fies e ere going to nee%. +oo-ing at the !o%e, its !ear that these three ines !onsist of essentiay the same statement being ee!&te% m&tipe times, ith some sight variations. $his i%ea of repetitionith variation is in!re%iby !ommon in programming probems, an% #ython has b&it in toos for epressing it 5 loops.
HG
Chapter B: +ists an% oops
4reating lists an" retrieving elements $o ma-e a ne ist, e p&t severa strings or n&mbers 1 insi%e s<&are bra!-ets, separate% by !ommas: apes B $*omo sapiens*, *Pan troglodytes*, *%orilla gorilla* consered/sites B $2', ", 132
)a!h in%ivi%&a item in a ist is !ae% an element . $o get a singe eement from the ist, rite the variabe name fooe% by the in"e! of the eement yo& ant in s<&are bra!-ets: apes B $*omo sapiens*, *Pan troglodytes*, *%orilla gorilla* consered/sites B $2', ", 132 print(apes$0# first/site B consered/sites$2
'f e ant to go in the other %ire!tion 5 i.e. e -no hi!h eement e ant b&t e %ont -no the in%e 5 e !an &se the index metho%: apes B $*omo sapiens*, *Pan troglodytes*, *%orilla gorilla* chimp/index B apes.index(*Pan troglodytes*# 4 chimp/index is no 1
emember that in #ython e start !o&nting from ero rather than one, so the first eement of a ist is aays at in%e ero. 'f e give a negative n&mber, #ython starts !o&nting from the end of the ist rather than the beginning 5 so its easy to get the ast eement from a ist: last/ape B apes$F1
1
?r in fa!t, any other type of va&e or variabe
HH
Chapter B: +ists an% oops
hat if e ant to get more than one eement from a istE e !an give a start an% stop position, separate% by a !oon, to spe!ify a range of eements: ran:s B $*:ingdom*,*phylum*, *class*, *order*, *family* loer/ran:s B ran:s$2! 4 loer ran:s are class, order and family
Does this oo- famiiarE 'ts the ea!t same notation that e &se% to get s&bstrings ba!- in !hapter 2, an% it or-s in ea!ty the same ay 5 n&mbers are inclusive at the start an% exclusive at the en%. $he fa!t that e &se the same notation for strings an% ists hints at a %eeper reationship beteen the to types. 'n fa!t, hat e ere %oing hen etra!ting s&bstrings in !hapter 2 as treating a string as t+oug+ it ere a list of c+aracters. $his i%ea 5 that e !an treat a variabe as tho&gh it ere a ist hen its not 5 is a poerf& one in #ython an% e !ome ba!to it ater in this !hapter.
Working with list elements $o a%% another eement onto the en% of an eisting ist, e !an &se the append metho%: apes B $*omo sapiens*, *Pan troglodytes*, *%orilla gorilla* apes.append(*Pan paniscus*#
append is an interesting metho% be!a&se it a!t&ay !hanges the variabe on hi!h its &se% 5 in the above eampe, the apes ist goes from having three eements to having fo&r. e !an get the ength of a ist by &sing the len f&n!tion, 9&st i-e e %i% for strings:
HI
Chapter B: +ists an% oops
apes B $*omo sapiens*, *Pan troglodytes*, *%orilla gorilla* print(*)here are * C str(len(apes## C * apes*# apes.append(*Pan paniscus*# print(*
$he o&tp&t shos that the n&mber of eements in apes reay has !hange%: )here are 3 apes
e !an !on!atenate to ists 9&st as e %i% ith strings, by &sing the p&s symbo: apes B $*omo sapiens*, *Pan troglodytes*, *%orilla gorilla* mon:eys B $*Papio ursinus*, *acaca mulatta* primates B apes C mon:eys print(str(len(apes## C * apes*# print(str(len(mon:eys## C * mon:eys*# print(str(len(primates## C * primates*#
4s e !an see from the o&tp&t, this %oesnt !hange either of the to origina ists 5 it ma-es a bran% ne ist hi!h !ontains eements from both: 3 apes 2 mon:eys primates
'f e ant to a%% eements from a ist onto the en% of an eisting ist, !hanging it in the pro!ess, e !an &se the extend metho%. extend behaves i-e append b&t ta-es a list as its arg&ment rather than a singe element . 7ere are to more ist metho%s that !hange the variabe theyre &se% on: reerse an% sort. (oth reerse an% sort or- by !hanging the or%er of the eements in
H@
Chapter B: +ists an% oops
the ist. 'f e ant to print o&t a ist to see ho this or-s, e nee% to &se% str 9&st as e %i% hen printing o&t n&mbers=: ran:s B $*:ingdom*,*phylum*, *class*, *order*, *family* print(*at the start ! * C str(ran:s## ran:s.reerse(# print(*after reersing ! * C str(ran:s## ran:s.sort(# print(*after sorting ! * C str(ran:s##
'f e ta-e a oo- at the o&tp&t, e !an see ho the or%er of the eements in the ist is !hange% by these to metho%s: at the start ! $:ingdom, phylum, class, order, family after reersing ! $family, order, class, phylum, :ingdom after sorting ! $class, family, :ingdom, order, phylum
(y %efa&t, #ython sorts strings in aphabeti!a or%er an% n&mbers in as!en%ing n&meri!a or%er1.
Writing a loop 'magine e ante% to ta-e o&r ist of apes: apes B $*omo sapiens*, *Pan troglodytes*, *%orilla gorilla*
an% print o&t ea!h eement on a separate ine, i-e this: omo sapiens is an ape Pan troglodytes is an ape %orilla gorilla is an ape
1
e !an sort in other ays too, b&t thats beyon% the s!ope of this boo-
I0
Chapter B: +ists an% oops
?ne ay to %o it o&% be to 9&st print ea!h eement separatey: print(apes$0 C * is an ape*# print(apes$1 C * is an ape*# print(apes$2 C * is an ape*#
b&t this is very repetitive an% reies on &s -noing the n&mber of eements in the ist. hat e nee% is a ay to say something aong the ines of > for each element in the list of apes0 print out the element0 followe" by the wor"s 7 is an ape7>. #ythons oop synta aos &s to epress those instr&!tions i-e this: for ape in apes! print(ape C * is an ape*#
+ets ta-e a moment to oo- at the %ifferent parts of this oop. e start by riting for x in y, here y is the name of the ist e ant to pro!ess an% x is the name e ant to &se for the !&rrent eement ea!h time ro&n% the oop.
x is 9&st a variabe name so it foos a the r&es that eve area%y earne% abo&t variabe names=, b&t it behaves sighty %ifferenty to a the other variabes eve seen so far. 'n a previo&s eampes, e !reate a variabe an% store something in it, an% then the va&e of that variabe %oesnt !hange &ness e !hange it o&rseves. 'n !ontrast, hen e !reate a variabe to be &se% in a oop, e %ont set its va&e 5 the va&e of the variabe i be a&tomati!ay set to ea!h eement of the ist in t&rn, an% it i be %ifferent ea!h time ro&n% the oop. 'mportanty, the oop variabe x ony eists insi%e the oop 5 it gets !reate% at the start of ea!h oop iteration, an% %isappears at the en%. $his means that on!e the oop has finishe% r&nning for the ast time, that variabe is gone forever. hen a variabe is restri!te% to a bo!- of !o%e i-e this, e !a it the variabes scope 5 e i see severa more eampes ater in the boo-.
I1
Chapter B: +ists an% oops
$his first ine of the oop en%s ith a !oon, an% a the s&bse<&ent ines 9&st one, in this !ase= are in%ente%. 'n%ente% ines !an start ith any n&mber of tab or spa!e !hara!ters, b&t they m&st a be in%ente% in the same ay. $his pattern 5 a ine hi!h en%s ith a !oon, fooe% by some in%ente% ines 5 is very !ommon in #ython, an% e see it in severa more pa!es thro&gho&t this boo-. 4 gro&p of in%ente% ines is often !ae% a block of !o%e1. 'n this !ase, e refer to the in%ente% bo!- as the bo"y of the oop, an% the ines insi%e it i be ee!&te% on!e for ea!h eement in the ist. $o refer to the !&rrent eement, e &se the variabe name that e rote in the first ine. $he bo%y of the oop !an !ontain as many ines as e i-e, an% !an in!&%e a the f&n!tions an% metho%s that eve earne% abo&t, ith one important e!eption: ere not aoe% to !hange the ist hie insi%e the bo%y of the oop 2. 7eres an eampe of a oop ith a more !ompi!ate% bo%y: apes B $*omo sapiens*, *Pan troglodytes*, *%orilla gorilla* for ape in apes! name/length B len(ape# first/letter B ape$0 print(ape C * is an ape. Lts name starts ith * C first/letter# print(*Lts name has * C str(name/length# C * letters*#
$he bo%y of the oop in the !o%e above has fo&r statements, to of hi!h are print statements, so ea!h time ro&n% the oop e get to ines of o&tp&t. 'f e oo- at the o&tp&t e !an see a si ines:
1 2
'f yo&re famiiar ith any other programming ang&ages, yo& might -no !o%e bo!-s as things that are s&rro&n%e% ith !&ry bra!-ets 5 the in%entation %oes the same 9ob in #ython Changing the ist hie ooping !an !a&se #ython to be!ome !onf&se% abo&t hi!h eements have area%y been pro!esse% an% hi!h are yet to !ome.
I2
omo sapiens is Lts name has 12 Pan troglodytes Lts name has 1 %orilla gorilla Lts name has 1
Chapter B: +ists an% oops
an ape. Lts name starts ith letters is an ape. Lts name starts ith P letters is an ape. Lts name starts ith % letters
hy is the above approa!h better than printing o&t these si ines in si separate statementsE e, for one thing, theres m&!h ess re%&n%an!y 5 here e ony nee%e% to rite to print statements. $his aso means that if e nee% to ma-e a !hange to the !o%e, e ony have to ma-e it on!e rather than three separate times. 4nother benefit of &sing a oop here is that if e ant to a%% some eements to the ist, e %ont have to to&!h the oop !o%e at a. Conse<&enty, it %oesnt matter ho many eements are in the ist, an% its not a probem if e %ont -no ho many are going to be in it at the time hen e rite the !o%e.
n"entation errors *nfort&natey, intro%&!ing toos i-e oops that re<&ire an in%ente% bo!- of !o%e aso intro%&!es the possibiity of a ne type of error 5 an ;ndentationError . ;oti!e hat happens hen the in%entation of one of the ines in the bo!- %oes not mat!h the others: apes B $*omo sapiens*, *Pan troglodytes*, *%orilla gorilla* for ape in apes! name/length B len(ape# first/letter B ape$0 print(ape C * is an ape. Lts name starts ith * C first/letter# print(*Lts name has * C str(name/length# C * letters*#
hen e r&n this !o%e, e get an error message before the program even starts to r&n:
I3
Chapter B: +ists an% oops
Lndentation9rror! unindent does not match any outer indentation leel
hen yo& en!o&nter an ;ndentationError , go ba!- to yo&r !o%e an% %o&be !he!- that a the ines in the bo!- mat!h &p. 4so %o&be!he!- that yo& are &sing either tabs or spa!es for in%entation, not bot+. $he easiest ay to %o this, as mentione% in !hapter 1, is to enabe tab emulation in yo&r tet e%itor.
,sing a string as a list eve area%y seen ho a string !an preten% to be a ist 5 e !an &se ist in%e notation to get in%ivi%&a !hara!ters or s&bstrings from insi%e a string. Can e aso &se oop notation to pro!ess a string as tho&gh it ere a istE es 5 if e rite a oop statement ith a string in the position here e% normay fin% a ist, #ython treats eac+ c+aracter in the string as a separate eement. $his aos &s to very easiy pro!ess a string one !hara!ter at a time: name B *martin* for character in name! print(*one character is * C character#
'n this !ase, ere 9&st printing ea!h in%ivi%&a !hara!ter: one one one one one one
character character character character character character
is is is is is is
m a r t i n
$he pro!ess of repeating a set of instr&!tions for ea!h eement of a ist or !hara!ter in a string= is !ae% iteration, an% e often ta- abo&t iterating over a ist or string.
IB
Chapter B: +ists an% oops
&plitting a string to make a list "o far in this !hapter, a o&r ists have been ritten man&ay. 7oever, there are penty of f&n!tions an% metho%s in #ython that pro%&!e ists as their o&tp&t. ?ne s&!h metho% that is parti!&ary interesting to bioogists is the split metho% hi!h or-s on strings. split ta-es a singe arg&ment, !ae% the "elimiter , an% spits the origina string herever it sees the %eimiter, pro%&!ing a ist. 7eres an eampe: names B *melanogaster,simulans,ya:u-a,ananassae* species B names.split(*,*# print(str(species##
e !an see from the o&tp&t that the string has been spit herever there as a !omma eaving &s ith a ist of strings: $melanogaster, simulans, ya:u-a, ananassae
?f !o&rse, on!e eve !reate% a ist in this ay e !an iterate over it &sing a oop, 9&st i-e any other ist.
terating over lines in a file 4nother very &sef& thing that e !an iterate over is a fie. J&st as a string !an preten% to be a ist for the p&rposes of ooping, a fie ob9e!t !an %o the same tri!- 1. hen e treat a string as a ist, ea!h !hara!ter be!omes an in%ivi%&a eement, b&t hen e treat a fie as a ist, ea!h line be!omes an in%ivi%&a eement. $his ma-es pro!essing a fie inebyine very easy:
1
'f yo&re intereste% in ho this >preten%ing> a!t&ay or-s, oo- &p the #ython %o!&mentation for iterators 5 b&t be prepare% to %o <&ite a bit of rea%ing8
I
Chapter B: +ists an% oops
file B open(*some/input.txt*# for line in file! 4 do something ith the line
4 <&i!- arning: hen yo&re riting a program that rea%s %ata from a fie, its best to &se eit+er the read metho% to store the entire !ontents in a variabe= or the oop metho% to %ea ith ea!h ine separatey=. 'f yo& try to mi them, yo& might get &nepe!te% behavio&r. $he reason for this is that #ython -eeps tra!- of its position in ea!h fie, so if yo& rea% the !ontents of a fie ob9e!t &sing the read metho%, an% then ater try to pro!ess it one ine at a time ith a oop, yo& ont get any inp&t be!a&se #ython thin-s its area%y at the en% of the fie. 'f yo& abso&tey have to &se one metho% an% then the other, yo& !an get ro&n% this probem by !osing an% then reopening the fie.
6ooping with ranges "ometimes e ant to oop over a ist of n&mbers. 'magine e have a protein se<&en!e: protein B *lspad:tn*
an% e ant to print o&t the first three resi%&es, then the first fo&r resi%&es, et!. et!.: ls lsp lspa lspad ...etc...
?ne ay to ta!-e the probem o&% be to &se a oop 5 e !o&% etra!t a s&bstring from the protein se<&en!e an% print it in the bo%y of the oop, an% the ony thing
IG
Chapter B: +ists an% oops
that o&% nee% to !hange is the stop position in the s&bstring. (&t hat are e going to iterate overE e !ant 9&st iterate over the protein string, be!a&se that i give &s in%ivi%&a resi%&es, hi!h is not hat e ant. e !an man&ay assembe a ist of stop positions, an% oop over that: stop/positions B $3,',,",7,G,J,10 for stop in stop/positions! su-string B protein$0!stop print(su-string#
b&t this seems !&mbersome, an% ony or-s if e -no the ength of the protein se<&en!e in a%van!e. 4 better so&tion is to &se the range f&n!tion. range is a b&itin #ython f&n!tion that generates ists of n&mbers for &s to oop over. $he behavio&r of the range f&n!tion %epen%s on ho many arg&ments e give it. (eo are a fe eampes, ith the o&tp&t fooing %ire!ty after the !o%e. ith a singe arg&ment, range i !o&nt &p from ero to that n&mber, e!&%ing the n&mber itsef: for num-er in range("#! print(num-er#
0 1 2 3 '
IH
Chapter B: +ists an% oops
ith to n&mbers, range i !o&nt &p from the first n&mber in!&sive 1= to the se!on% e!&sive=: for num-er in range(3, G#! print(num-er#
3 ' " 7
ith three n&mbers, range i !o&nt &p from the first to the se!on% ith the step sie given by the thir%: for num-er in range(2, 1', '#! print(num-er#
2 " 10
(ecap 'n this !hapter eve seen severa toos that or- together to ao o&r programs to %ea eeganty ith m&tipe pie!es of %ata. +ists et &s store many eements in a singe variabe, an% oops et &s pro!ess those eements one by one. 'n earning abo&t oops, eve aso been intro%&!e% to the bo!- synta an% the importan!e of in%entation in #ython. 1
$he r&es for ranges are the same as for array notation 5 in!&sive on the o en%, e!&sive on the high en% 5 so yo& ony have to memorie them on!e8
II
Chapter B: +ists an% oops
eve aso seen severa &sef& ays in hi!h e !an &se the notation eve earne% for or-ing ith ists ith other types of %ata. Depen%ing on the !ir!&mstan!es, e !an &se strings, files, an% ranges as if they ere ists. $his is a very hepf& feat&re of #ython, be!a&se on!e eve be!ome famiiar ith the synta for or-ing ith ists, e !an &se it in many %ifferent pa!e. +earning abo&t these toos has aso hepe% &s ma-e sense of some interesting behavio&r that e observe% in earier !hapters.
I@
Chapter B: +ists an% oops
!ercises .ote: a the fies mentione% in these eer!ises !an be fo&n% in the chapter* fo%er of the eer!ises %onoa%.
Processing '.4 in a file $he fie input=t!t !ontains a n&mber of D;4 se<&en!es, one per ine. )a!h se<&en!e starts ith the same 1B base pair fragment 5 a se<&en!ing a%apter that sho&% have been remove%. rite a program that i a= trim this a%apter an% rite the !eane% se<&en!es to a ne fie an% b= print the ength of ea!h se<&en!e to the s!reen.
ultiple exons from genomic '.4 $he fie genomic"na=t!t !ontains a se!tion of genomi! D;4, an% the fie e!ons=t!t !ontains a ist of start/stop positions of eons. )a!h eon is on a separate ine an% the start an% stop positions are separate% by a !omma. rite a program that i etra!t the eon segments, !on!atenate them, an% rite them to a ne fie.
@0
Chapter B: +ists an% oops
&olutions Processing '.4 in a file $his seems a bit more !ompi!ate% than previo&s eer!ises 5 e are being as-e% to rite a program that %oes to things at on!e8 5 so ets ta!-e it one step at a time. First, e rite a program that simpy rea%s ea!h se<&en!e from the fie an% prints it to the s!reen: file B open(*input.txt*# for dna in file! print(dna#
e !an see from the o&tp&t that eve forgotten to remove the neines from the en%s of the D;4 se<&en!es 5 there is a ban- ine beteen ea!h: A))&%A))A)AA%&)&%A)&%A)&%A)&%A)&%A)&%A)&%A)&%A)&%A)&%A)& A))&%A))A)AA%&A&)%A)&%A)&%A)&%A)&%A)&%A)%&)A)&%)&%) A))&%A))A)AA%&A)&%A)&A&%A)&)A)&%)A&%)A)%&A)A)&%A)A)&%A)&%)A%)& A))&%A))A)AA%&A&)A)&%A)%A)&)A%&)A&%A)&%)A%&)%)A A))&%A))A)AA%&A&)A%&)A%)&)&%A)%&A)%A)&A%&))A%&)%A)%A)%&)A)%&A
b&t e ignore that for no. $he net step is to remove the first 1B bases of ea!h se<&en!e. e -no that e ant to ta-e a s&bstring from ea!h se<&en!e, starting at the fifteenth !hara!ter, an% !ontin&ing to the en%. *nfort&natey, the se<&en!es are a %ifferent engths, so the stop position is going to be %ifferent for a of them. e have to !a!&ate the position of the ast !hara!ter for ea!h se<&en!e, by &sing the len f&n!tion to !a!&ate the ength.
@1
Chapter B: +ists an% oops
7eres hat the !o%e oo-s i-e ith the s&bstring part a%%e%: file B open(*input.txt*# for dna in file! last/character/position B len(dna# trimmed/dna B dna$1'!last/character/position print(trimmed/dna#
4s before, e are simpy printing the trimme% D;4 se<&en!e to the s!reen, an% from the o&tp&t e !an !onfirm that the first 1B bases have been remove% from ea!h se<&en!e: )&%A)&%A)&%A)&%A)&%A)&%A)&%A)&%A)&%A)&%A)& A&)%A)&%A)&%A)&%A)&%A)&%A)%&)A)&%)&%) A)&%A)&A&%A)&)A)&%)A&%)A)%&A)A)&%A)A)&%A)&%)A%)& A&)A)&%A)%A)&)A%&)A&%A)&%)A%&)%)A A&)A%&)A%)&)&%A)%&A)%A)&A%&))A%&)%A)%A)%&)A)%&A
;o that e -no o&r !o%e is or-ing, e sit!h from printing to the s!reen to riting to a fie. e have to open the fie before the oop, then rite the trimme% se<&en!es to the fie inside the oop: file B open(*input.txt*# output B open(*trimmed.txt*, **# for dna in file! last/character/position B len(dna# trimmed/dna B dna$1'!last/character/position output.rite(trimmed/dna#
@2
Chapter B: +ists an% oops
?pening &p the trimme"=t!t fie, e !an see that the res&t oo-s goo%. 't %i%nt matter that e never remove% the neines, be!a&se they appear in the !orre!t pa!e in the o&tp&t fie anyay: )&%A)&%A)&%A)&%A)&%A)&%A)&%A)&%A)&%A)&%A)& A&)%A)&%A)&%A)&%A)&%A)&%A)%&)A)&%)&%) A)&%A)&A&%A)&)A)&%)A&%)A)%&A)A)&%A)A)&%A)&%)A%)& A&)A)&%A)%A)&)A%&)A&%A)&%)A%&)%)A A&)A%&)A%)&)&%A)%&A)%A)&A%&))A%&)%A)%A)%&)A)%&A
;o the fina step 5 printing the engths to the s!reen 5 re<&ires 9&st one more ine of !o%e. 7eres the fina program in f&, ith !omments: 4 open the input file file B open(*input.txt*# 4 open the output file output B open(*trimmed.txt*, **# 4 go through the input file one line at a time for dna in file! 4 calculate the position of the last character last/character/position B len(dna# 4 get the su-string from the 1th character to the end trimmed/dna B dna$1'!last/character/position 4 print out the trimmed seuence output.rite(trimmed/dna# 4 print out the length to the screen print(*processed seuence ith length * C str(len(trimmed/dna###
@3
Chapter B: +ists an% oops
ultiple exons from genomic '.4 $his is very simiar to the eer!ises from the previo&s to !hapters, an% so o&r so&tion to it is going to oo- very simiar. +ets !on!entrate on the ne bit of the probem first 5 rea%ing the fie of eon o!ations. 4s before, e !an start by opening &p the fie an% printing ea!h ine to the s!reen: exon/locations B open(*exons.txt*# for line in exon/locations! print(line#
$his gives &s a oop in hi!h e are %eaing ith a %ifferent eon ea!h time ro&n%. 'f e oo- at the o&tp&t, e !an see that e sti have a neine at the en% of ea!h ine, b&t e not orry abo&t that for no: ,G 72,133 1J0,27" 3'0,3JG
;o e have to spit &p ea!h ine into a start an% stop position. $he split metho% is probaby a goo% !hoi!e for this 9ob 5 ets see hat happens hen e spit ea!h ine &sing a !omma as the %eimiter: exon/locations B open(*exons.txt*# for line in exon/locations! positions B line.split(,# print(positions#
$he o&tp&t shos that ea!h ine, hen spit, t&rns into a ist of to eements:
@B
Chapter B: +ists an% oops
$, Gn $72, 133n $1J0, 27"n $3'0, 3JGn
$he se!on% eement of ea!h ist has a neine on the en%, be!a&se e havent remove% them. +ets try assigning the start an% stop position to sensibe variabe names, an% printing them o&t in%ivi%&ay: exon/locations B open(*exons.txt*# for line in exon/locations! positions B line.split(,# start B positions$0 stop B positions$1 print(*start is * C start C *, stop is * C stop#
$he o&tp&t shos that this approa!h or-s 5 the start an% stop variabes ta-e %ifferent va&es ea!h time ro&n% the oop: start is , stop is G start is 72, stop is 133 start is 1J0, stop is 27" start is 3'0, stop is 3JG
;o ets try p&tting these variabes to &se. e rea% the genomi! se<&en!e from the fie a in one go &sing read 5 theres no nee% to pro!ess ea!h ine separatey, as e 9&st ant the entire !ontents. $hen e &se the eon !oor%inates to etra!t one eon ea!h time ro&n% the oop, an% print it to the s!reen:
@
1 2 3 4 5 6 7 8
Chapter B: +ists an% oops
genomic/dna B open(*genomic/dna.txt*#.read(# exon/locations B open(*exons.txt*# for line in exon/locations! positions B line.split(,# start B positions$0 stop B positions$1 exon B genomic/dna$start!stop print(*exon is! * C exon#
*nfort&natey, hen e r&n this !o%e e get an error at ine H: ile *multiple/exons/from/genomic/dna.py*, line 7, in ;module+ exon B genomic/dna$start!stop )ype9rror! slice indices must -e integers or
hat has gone rongE e!a that the res&t of &sing split on a string is a ist of strings 5 this means that the start an% stop aria$les in o&r program are aso strings be!a&se theyre 9&st in%ivi%&a eements of the positions ist=. $he probem !omes hen e try to &se them as n&mbers in ine H. Fort&natey, its easiy fie% 5 e 9&st have to &se the int f&n!tion to t&rn o&r strings into n&mbers: start B int(positions$0# stop B int(positions$1#
an% the program or-s as inten%e%. ;et step: %oing something &sef& ith the eons, rather than 9&st printing them to the s!reen. $he eer!ise %es!ription says that e have to !on!atenate the eon se<&en!es to ma-e a ong !o%ing se<&en!e. 'f e ha% a the eons in separate variabes, then this o&% be easy coding/se B exon1 C exon2 C exon3 C exon'
@G
Chapter B: +ists an% oops
b&t instea% e have a singe exon variabe that stores one eon at a time. 7eres one ay to get the !ompete !o%ing se<&en!e: before the oop starts e !reate a ne variabe !ae% coding_seAuence an% assign it to an empty string. $hen, ea!h time ro&n% the oop, e a%% the !&rrent eon on to the en%, an% store the res&t ba!- in the same variabe. hen the oop has finishe%, the variabe i !ontain a the eons. $his is hat the !o%e oo-s i-e ith ine n&mbers as the program is getting <&ite ong=: 1 2 3 4 5 6 7 8 9 10
genomic/dna B open(*genomic/dna.txt*#.read(# exon/locations B open(*exons.txt*# coding/seuence B ** for line in exon/locations! positions B line.split(,# start B int(positions$0# stop B int(positions$1# exon B genomic/dna$start!stop coding/seuence B coding/seuence C exon print(*coding seuence is ! * C coding/seuence#
?n ine 3 e !reate the coding_seAuence variabe, an% on ine @, insi%e the oop, e a%% the !&rrent exon on to the en%. $his is an &n&s&a type of variabe assignment, be!a&se the coding_seAuence variabe is on both the eft an% right si%e of the e<&as sign. $he tri!- to &n%erstan%ing ine @ is to rea% the righthan% si%e of the statement first i.e. >concatenate the current coding_seAuence an" the current exon 0 then store the result of that concatenation in coding_seAuence>. ?n ine 10, instea% of printing the eon, ere printing the !o%ing se<&en!e, an% e !an see from the o&tp&t ho the !o%ing se<&en!e is gra%&ay b&it &p as e go ro&n% the oop:
@H
Chapter B: +ists an% oops
coding seuence is ! &%)A&&%)&%A&%A)%&)A&%A)&%)&%A)&%)A%)&%A)&A)&%A)&%A)&% coding seuence is ! &%)A&&%)&%A&%A)%&)A&%A)&%)&%A)&%)A%)&%A)&A)&%A)&%A)&%&%A)&%A)&%A)A)&%A)&%A )A)&A)&%A)%&A)&%A)&A)&%A)&%A)&%A)&%A)&%A coding seuence is ! &%)A&&%)&%A&%A)%&)A&%A)&%)&%A)&%)A%)&%A)&A)&%A)&%A)&%&%A)&%A)&%A)A)&%A)&%A )A)&A)&%A)%&A)&%A)&A)&%A)&%A)&%A)&%A)&%A&%A)&%A)&%A)&%)A%&)A%&)A%&)A%A)&%A )&A)&A)&%)A%&)A%&)&%A&)A%&)A&%)A&%A)&%A)%&A)&%A)&%)A coding seuence is ! &%)A&&%)&%A&%A)%&)A&%A)&%)&%A)&%)A%)&%A)&A)&%A)&%A)&%&%A)&%A)&%A)A)&%A)&%A )A)&A)&%A)%&A)&%A)&A)&%A)&%A)&%A)&%A)&%A&%A)&%A)&%A)&%)A%&)A%&)A%&)A%A)&%A )&A)&A)&%)A%&)A%&)&%A&)A%&)A&%)A&%A)&%A)%&A)&%A)&%)A&%A)&%A)&%A)&%A)&%A)&% A)&%A)&%A)&%A)&%A)&%)A%&)A%&)A&%A)&%
$he fina step is to save the !o%ing se<&en!e to a fie. e !an %o this at the en% of the program ith three ines of !o%e. 7eres the fina !o%e ith !omments:
@I
Chapter B: +ists an% oops
4 open the genomic dna file and read the contents genomic/dna B open(*genomic/dna.txt*#.read(# 4 open the exons locations file exon/locations B open(*exons.txt*# 4 create a aria-le to hold the coding seuence coding/seuence B ** 4 go through each line in the exon locations file for line in exon/locations! 4 split the line using a comma positions B line.split(,# 4 get the start and stop positions start B int(positions$0# stop B int(positions$1# 4 extract the exon from the genomic dna exon B genomic/dna$start!stop 4 append the exon to the end of the current coding seuence coding/seuence B coding/seuence C exon 4 rite the coding seuence to an output file output B open(*coding/seuence.txt*, **# output.rite(coding/seuence# output.close(#
@@
Chapter : riting o&r on f&n!tions
!:
%riting our on functions
Why "o we want to write our own functions? $a-e a oo- ba!- at the very first eer!ise in this boo- 5 the one in !hapter 2 here e ha% to rite a program to !a!&ate the 4$ !ontent of a D;4 se<&en!e. +ets remin% o&rseves of the !o%e: 1 2 3 4 5 6
my/dna B *A&)%A)&%A))A&%)A)A%)A)))%&)A)&A)A&A)A)A)A)&%A)%&%))&A)* length B len(my/dna# a/count B my/dna.count(A# t/count B my/dna.count()# at/content B (a/count C t/count# length print(*A) content is * C str(at/content##
'f e %is!o&nt the first ine hose 9ob is to store the inp&t se<&en!e= an% the ast ine hose 9ob is to print the res&t=, e !an see that it ta-es fo&r ines of !o%e to !a!&ate the 4$ !ontent 1. $his means that every pa!e in o&r !o%e here e ant to !a!&ate the 4$ !ontent of a se<&en!e, e nee% these same fo&r ines 5 an% e have to ma-e s&re e !opy them ea!ty, itho&t any mista-es. 't o&% be m&!h simper if #ython ha% a b&itin f&n!tion ets !a it get_at_content = for !a!&ating 4$ !ontent. 'f that ere the !ase, then e !o&% 9&st r&n get_at_content in the same ay e r&n print, or len, or open. 4tho&gh, sa%y, #ython %oes not have s&!h a b&itin f&n!tion, it %oes have the net best thing 5 a ay for &s to !reate o&r on f&n!tions. Creating o&r on f&n!tion to !arry o&t a parti!&ar 9ob has many benefits. 't aos &s to re&se the same !o%e many times ithin a program itho&t having to !opy it o&t ea!h time. 4%%itionay, if e fin% that e have to ma-e a !hange to the !o%e, 1
e !o&%, of !o&rse, !ompress this %on to a singe ine: at_content @ +my_dna.count+)!), F my_dna.count+)"),, G len+my_dna, b&t it o&% be m&!h ess rea%abe.
100
Chapter : riting o&r on f&n!tions
e ony have to %o it in one pa!e. "pitting o&r !o%e into f&n!tions aso aos &s to ta!-e arger probems, as e !an or- on %ifferent bits of the !o%e in%epen%enty. e !an aso re&se !o%e a!ross m&tipe programs.
efining a function +ets go ahea% an% !reate o&r get_at_content f&n!tion. (efore e start typing, e nee% to fig&re o&t hat the inp&ts the n&mber an% types of the function arguments= an% o&tp&ts the type of the return value= are going to be. For this f&n!tion, that seems pretty obvio&s 5 the inp&t is going to be a singe D;4 se<&en!e, an% the o&tp&t is going to be a %e!ima n&mber. $o transate these into #ython terms: the f&n!tion i ta-e a singe arg&ment of type string , an% i ret&rn a va&e of type number 1. 7eres the !o%e: def get/at/content(dna#! length B len(dna# a/count B dna.count(A# t/count B dna.count()# at/content B (a/count C t/count# length return at/content
Reminder: if yo&re &sing #ython 2 rather than #ython 3, in!&%e this ine at the top of yo&r program: from //future// import diision
$he first ine of the f&n!tion %efinition !ontains a severa %ifferent eements. e start ith the or% def, hi!h is short for "efine riting a f&n!tion is !ae% "efining it=. Fooing that e rite the name of the f&n!tion, fooe% by the names of the arg&ment variabes in parentheses. J&st i-e e sa before ith 1
'n fa!t, e !an be a itte bit more spe!ifi!: e !an say that the ret&rn va&e i be of type float 5 a foating point n&mber i.e. one ith a %e!ima point=.
101
Chapter : riting o&r on f&n!tions
norma variabes, the f&n!tion name an% the arg&ment names are arbitrary 5 e e !an &se hatever e i-e. $he first ine en%s ith a !oon, 9&st i-e i-e the first ine of the oops that e ere oo-ing at in the previo&s !hapter. 4n% 9&st i-e oops, this ine is fooe% by a block of in%ente% ines 5 the function bo"y . $he f&n!tion bo%y !an have as many ines of !o%e as e i-e, as ong as they a have the same in%entation. ithin ithin the f&n!tion bo%y, bo%y, e !an refer to the arg&ments by &sing the variabe names from the first ine. 'n this !ase, the variabe dna refers to the se<&en!e that as passe% in as the arg&ment to the f&n!tion. $he ast ine of the f&n!tion !a&ses it to ret&rn the 4$ !ontent !ontent that as !a!&ate% in the f&n!tion bo%y. $o return from a f&n!tion, e e simpy rite ret&rn fooe% by the va&e that the f&n!tion sho&% o&tp&t. $here are a !o&pe of important things to be aare of hen riting f&n!tions. Firsty, e nee% to ma-e a !ear %istin!tion beteen "efining a a f&n!tion, an% running it it e refer to r&nning a f&n!tion as calling it=. $he !o%e eve ritten above i not !a&se anything to happen hen e r&n it, be!a&se eve not a!t&ay as-e% #ython to ee!&te the get_at_content f&n!tion 5 e have simpy %efine% hat it is. $he !o%e in the f&n!tion f&n!tion i not not be ee!&te% &nti &nti e !a the f&n!tion f&n!tion i-e this: get/at/content(*A)%A&)%%A&&A*#
'f e simpy !a the f&n!tion i-e that, hoever, then the 4$ !ontent i vanish on!e its its been !a!&ate%. 'n 'n or%er to &se the f&n!tion to %o something &sef&, e m&st either store the res&t in a variabe: at/content B get/at/content(*A)%A&)%%A&&A*#
?r &se it %ire!ty:
102
Chapter : riting o&r on f&n!tions
print(*A) content is * C str(get/at/content(*A)%A&)%%A&&A*###
"e!on%y, its its important to &n%erstan% that the arg&ment variabe dna %oes not ho% any parti!&ar va&e hen the f&n!tion is %efine% 1. 'nstea%, its 9ob is to ho% hatever va&e va&e is given as the arg&ment hen hen the f&n!tion is !ae%. !ae%. 'n this ay its its anaogo&s to the oop variabes e sa in the previo&s pr evio&s !hapter: oop variabes ho% a %ifferent va&e ea!h time ro&n% the oop, an% an% f&n!tion arg&ment variabes ho% a %ifferent va&e ea!h time the f&n!tion is !ae%. Finay, Finay, be aare that the same s!oping r&es that appie% to t o oops aso appy to f&n!tions 5 any variabes that e !reate as part of the f&n!tion ony eist insi%e the f&n!tion, an% an% !annot be a!!esse% o&tsi%e. 'f e try tr y to &se a variabe thats !reate% insi%e the f&n!tion from o&tsi%e: def get/at/content(dna#! length B len(dna# a/count B dna.count(A# t/count B dna.count()# at/content B (a/count C t/count# length return at/content print(a/count#
e get an error:
1
'n%ee%, 'n%ee%, it %oesnt %oesnt a!t&ay a!t&ay eist eist hen hen its its %efine% %efine%,, ony ony hen hen it r&ns.
103
Chapter : riting o&r on f&n!tions
4alling an" improving our function +ets rite a sma program that &ses o&r ne f&n!tion, to see ho it or-s. e try both storing the res&t in a variabe before printing it ines I an% @= an% printing it %ire!ty ines 10 an% 11=: 1 2 3 4 5 6
def get/at/content(dna#! length B len(dna# a/count B dna.count(A# t/count B dna.count()# at/content B (a/count C t/count# length return at/content
7 8
my/at/content B get/at/content(*A)%&%&%A)&%A)&%AA)&%*# 9 print(str(my/at/content## 10 print(get/at/content(*A)%&A)%&AA&)%)A%&*## 11 print(get/at/content(*aactgtagctagctagcagcgta*##
+oo-ing at the o&tp&t, e !an see that the first f&n!tion !a or-s fine 5 the 4$ !ontent is !a!&ate% to be 0.B, is store% in the variabe myYatY!ontent, then printe%. 7oever, the o&tp&t for the net to !as is not so great. $he !a at ine 10 pro%&!es a n&mber ith ay too many fig&res after the %e!ima point, an% an% the !a at ine 11, ith the inp&t se<&en!e in oer !ase, gives a res&t of 0.0, hi!h is %efinitey not !orre!t: 0.' 0.2J'117"'70GG2' 0.0
e fi these probems by ma-ing a !o&pe of !hanges to the get_at_content f&n!tion. e e !an a%% a ro&n%ing step in or%er to imit the n&mber of signifi!ant fig&res in the res&t. #ython has a b&itin round f&n!tion that ta-es to arg&ments 5 the n&mber e ant to ro&n%, an% the n&mber of signifi!ant fig&res. e !a the round f&n!tion on the res&t before e ret&rn it. 4n% 4n% e !an fi the
10B
Chapter : riting o&r on f&n!tions
oer !ase probem by !onverting the inp&t se<&en!e to &pper !ase before starting the !a!&ation. 7eres 7eres the ne version of the f&n!tion, ith the same three f&n!tion !as: def get/at/content(dna#! length B len(dna# a/count B dna.upper(#.count(A# t/count B dna.upper(#.count()# at/content B (a/count C t/count# length return round(at/content, 2# my/at/content B get/at/content(*A)%&%&%A)&%A)&%AA)&%*# print(str(my/at/content## print(get/at/content(*A)%&A)%&AA&)%)A%&*## print(get/at/content(*aactgtagctagctagcagcgta*##
an% no the o&tp&t is 9&st as e ant: 0.' 0.3 0.2
e !an ma-e the t he f&n!tion even better tho&gh: hy not ao it to be !ae% ith the n&mber of signifi!ant fig&res as an arg&ment1E 'n the above !o%e, eve pi!-e% to signifi!ant fig&res, b&t b&t there might be sit&ations here e ant to see more. m ore. 4%%ing the se!on% arg&ment is easy e 9&st a%% it to the arg&ment variabe ist on the first ine of the f&n!tion %efinition, an% an% then &se the ne arg&ment variabe in the !a to round. e thro in a fe !as to the ne version of the f&n!tion ith %ifferent arg&ments to !he!- that it or-s:
1
4n even even better better so&tio so&tion n o&% o&% be to spe!ify spe!ify the the n&mber n&mber of signifi! signifi!ant ant fig&res fig&res in the the string string represent representation ation of the n&mber hen its printe%.
10
Chapter : riting o&r on f&n!tions
def get/at/content(dna, sig/figs#! length B len(dna# a/count B dna.upper(#.count(A# t/count B dna.upper(#.count()# at/content B (a/count C t/count# length return round(at/content, sig/figs# test/dna B *A)%&A)%&AA&)%)A%&* print(get/at/content(test/dna, 1## print(get/at/content(test/dna, 2## print(get/at/content(test/dna, 3##
$he o&tp&t !onfirms that the ro&n%ing or-s as inten%e%: 0. 0.3 0.2J
ncapsulation with functions +ets pa&se for a moment an% !onsi%er hat happene% in the previo&s se!tion. e rote a f&n!tion, an% then rote some !o%e that &se% that f&n!tion. 'n the pro!ess of riting the !o%e that &se% the f&n!tion, e %is!overe% a !o&pe of probems ith o&r origina f&n!tion %efinition. %e ere t+en able to go bac0 and c+ange t+e function definition) it+out +aving to ma0e an* c+anges to t+e code t+at used t+e function. 've ritten that ast senten!e in bo%, be!a&se its in!re%iby important. 'ts no eaggeration to say that &n%erstan%ing the impi!ations of that senten!e is the -ey to being abe to rite arger, &sef& programs. $he reason its so important is that it %es!ribes a programming phenomenon that e !a encapsulation. )n!aps&ation 9&st means %ivi%ing &p a !ompe program into itte bits hi!h e !an or- on in%epen%enty. 'n the eampe above, the !o%e is %ivi%e% into to parts 5 the part
10G
Chapter : riting o&r on f&n!tions
here e %efine the f&n!tion, an% the part here e &se it 5 an% e !an ma-e !hanges to one part itho&t orrying abo&t the effe!ts on the other part. $his is a very poerf& i%ea, be!a&se itho&t it, the sie of programs e !an rite is imite% to the n&mber of ines of !o%e e !an ho% in o&r hea% at one time. "ome of the eampe !o%e in the so&tions to eer!ises in the previo&s !hapter ere starting to p&sh at this imit area%y, even for reativey simpe probems. (y !ontrast, &sing f&n!tions aos &s to b&i% &p a !ompe program from sma b&i%ing bo!-s, ea!h of hi!h in%ivi%&ay is sma eno&gh to &n%erstan% in its entirety. (e!a&se &sing f&n!tions is so important, f&t&re so&tions to eer!ises i &se them hen appropriate, even hen its not epi!ity mentione% in the probem tet. ' en!o&rage yo& to get into the habit of &sing f&n!tions in yo&r so&tions too.
/unctions "on7t always have to take an argument $heres nothing in the r&es of #ython to say that yo&r f&n!tion must ta-e an arg&ment. 'ts perfe!ty possibe to %efine a f&n!tion ith no arg&ments: def get/a/num-er(#! return '2
b&t s&!h f&n!tions ten% not to be very &sef&. For eampe, e !an rite a version of get_at_content that %oesnt re<&ire any arg&ments by setting the va&e of the dna variabe insi%e the f&n!tion: def get/at/content(#! dna B *A&)%A)%&)A%&)A* length B len(dna# a/count B dna.upper(#.count(A# t/count B dna.upper(#.count()# at/content B (a/count C t/count# length return round(at/content, 2#
10H
Chapter : riting o&r on f&n!tions
b&t thats obvio&sy not very &sef&. ?!!asionay yo& may be tempte% to rite a noarg&ment f&n!tion that or-s i-e this: 1 2 3 4 5 6
def get/at/content(#! length B len(dna# a/count B dna.upper(#.count(A# t/count B dna.upper(#.count()# at/content B (a/count C t/count# length return round(at/content, 2#
7 8
dna B *A&)%A)&%A)&%* 9 print(get/at/content(##
4t first this seems i-e a goo% i%ea 5 it or-s be!a&se the f&n!tion gets the va&e of the dna variabe that is set on ine I 1. 7oever, this brea-s the en!aps&ation that e or-e% so har% to a!hieve. $he f&n!tion no ony or-s if there is a variabe !ae% dna set in the bit of the !o%e here the f&n!tion is !ae%, so the to pie!es of !o%e are no onger in%epen%ent. 'f yo& fin% yo&rsef riting !o%e i-e this, its &s&ay a goo% i%ea to i%entify hi!h variabes from o&tsi%e the f&n!tion are being &se% insi%e it, an% t&rn them into arg&ments.
1
't %oesnt matter that the variabe is set after the f&n!tion is %efine% 5 a that matters it that its set before the f&n!tion is !ae% on ine @.
10I
Chapter : riting o&r on f&n!tions
/unctions "on7t always have to return a value Consi%er this variation of o&r f&n!tion 5 instea% of returning the 4$ !ontent, this f&n!tion prints it to the s!reen: def print/at/content(dna#! length B len(dna# a/count B dna.upper(#.count(A# t/count B dna.upper(#.count()# at/content B (a/count C t/count# length print(str(round(at/content, 2###
hen yo& first start riting f&n!tions, its very tempting to %o this -in% of thing. o& thin- > >C0 nee" to calculate an" print the A' content < 7ll write a function that "oes both>. $he tro&be ith this approa!h is that it res&ts in a f&n!tion that is ess feibe. ight no yo& ant to print the 4$ !ontent to the s!reen, b&t hat if yo& ater %is!over that yo& ant to rite it to a fie, or &se it as part of some other !a!&ationE o& have to rite more f&n!tions to !arry o&t these tas-s. $he -ey to %esigning feibe f&n!tions is to re!ognie that the 9ob calculate an" print the A' content is a!t&ay to separate 9obs 5 !a!&ating the 4$ !ontent, an% printing it. $ry to rite yo&r f&n!tions in s&!h a ay that they 9&st %o one 9ob. o& !an then easiy rite !o%e to !arry o&t more !ompi!ate% 9obs by &sing yo&r simpe f&n!tions as b&i%ing bo!-s.
/unctions can be calle" with name" arguments hat %o e nee% to -no abo&t a f&n!tion in or%er to be abe to &se itE e nee% to -no hat the ret&rn va&e an% type is, an% e nee% to -no the n&mber an% type of the arg&ments. For the eampes eve seen so far in this boo-, e aso nee% to -no the order of the arg&ments. For instan!e, to &se the open f&n!tion e nee% to -no that the name of the fie !omes first, fooe% by the mo%e of the fie. 4n% to &se o&r toarg&ment version of get_at_content as %es!ribe% above, e nee%
10@
Chapter : riting o&r on f&n!tions
to -no that the D;4 se<&en!e !omes first, fooe% by the n&mber of signifi!ant fig&res. $heres a feat&re in #ython !ae% keywor" arguments hi!h aos &s to !a f&n!tions in a sighty %ifferent ay. 'nstea% of giving a ist of arg&ments in parentheses: get/at/content(*A)&%)%A&)&%*, 2#
e !an s&ppy a ist of arg&ment variabe names an% va&es 9oine% by e<&as signs: get/at/content(dnaB*A)&%)%A&)&%*, sig/figsB2#
$his stye of !aing f&n!tions 1 has severa a%vantages. 't %oesnt rey on the or%er of arg&ments, so e !an &se hi!hever or%er e prefer. $hese to statements behave i%enti!ay: get/at/content(dnaB*A)&%)%A&)&%*, sig/figsB2# get/at/content(sig/figsB2, dnaB*A)&%)%A&)&%*#
'ts aso !earer to rea% hats happening hen the arg&ment names are given epi!ity. e !an even mi an% mat!h the to styes of !aing 5 the fooing are a i%enti!a: get/at/content(*A)&%)%A&)&%*, 2# get/at/content(dnaB*A)&%)%A&)&%*, sig/figsB2# get/at/content(*A)&%)%A&)&%*, sig/figsB2#
4tho&gh ere not aoe% to start off ith -eyor% arg&ments then sit!h ba!to norma 5 this i !a&se an error: 1
't or-s ith metho%s too, in!&%ing a the ones eve seen so far.
110
Chapter : riting o&r on f&n!tions
get/at/content(dnaB*A)&%)%A&)&%*, 2#
Keyor% arg&ments !an be parti!&ary &sef& for f&n!tions an% metho%s that have a ot of arg&ments, an% e &se them here appropriate in the eampes an% eer!ise so&tions in the rest of this boo-.
/unction arguments can have "efaults eve en!o&ntere% f&n!tion arg&ments ith %efa&ts before, hen e ere %is!&ssing opening fies in !hapter 3. e!a that the open f&n!tion ta-es to arg&ments 5 a fie name an% a mo%e string 5 b&t that if e !a it ith ust a fie name it &ses a %efa&t va&e for the mo%e string. e !an easiy ta-e a%vantage of this feat&re in o&r on f&n!tions 5 e simpy spe!ify the %efa&t va&e in the first ine of the f&n!tion %efinition. 7eres a version of o&r get_at_content f&n!tion here the %efa&t n&mber of signifi!ant fig&res is to: def get/at/content(dna, sig/figsB2#! length B len(dna# a/count B dna.upper(#.count(A# t/count B dna.upper(#.count()# at/content B (a/count C t/count# length return round(at/content, sig/figs#
;o e have the best of both or%s. 'f the f&n!tion is !ae% ith to arg&ments, it i &se the n&mber of signifi!ant fig&res spe!ifie% if its !ae% ith one arg&ment, it i &se the %efa&t va&e of to signifi!ant fig&res. +ets see some eampes: get/at/content(*A)&%)%A&)&%*# get/at/content(*A)&%)%A&)&%*, 3# get/at/content(*A)&%)%A&)&%*, sig/figsB'#
111
Chapter : riting o&r on f&n!tions
$he f&n!tion ta-es !are of fiing in the %efa&t va&e for sig_figs for the first f&n!tion !a here none is s&ppie%: 0.' 0.' 0.''
F&n!tion arg&ment %efa&ts ao &s to rite very feibe f&n!tions hi!h !an have varying n&mbers of arg&ments. 't ony ma-es sense to &se them for arg&ments here a sensibe %efa&t !an be !hosen 5 theres no point spe!ifying a %efa&t for the dna arg&ment in o&r eampe. $hey are parti!&ary &sef& for f&n!tions here some of the options are ony going to be &se% infre<&enty.
'esting functions hen riting !o%e of any type, its important to perio%i!ay !he!- that yo&r !o%e %oes hat yo& inten% it to %o. 'f yo& oo- ba!- over the so&tions to eer!ises from the first fe !hapters, yo& !an see that e generay test o&r !o%e at ea!h step by printing some o&tp&t to the s!reen an% !he!-ing that it oo-s ?K. For eampe, in !hapter 2 hen e ere first !a!&ating 4$ !ontent, e &se% a very short test se<&en!e to verify that o&r !o%e or-e% before r&nning it on the rea inp&t. $he reason e &se% a test se<&en!e as that, be!a&se it as so short, e !o&% easiy or- o&t the anser by eye an% !ompare it to the anser given by o&r !o%e. $his i%ea 5 r&nning !o%e on a test inp&t an% !omparing the res&t to an anser t+at e 0no to be correct 1 5 is s&!h a &sef& one that #ython has a b&itin too for epressing it: assert. 4n assertion !onsists of the or% assert, fooe% by a !a to o&r f&n!tion, then to e<&as signs, then the res&t that e epe!t 2.
1 2
$hin- of it as simiar to r&nning a positive !ontro in a etab eperiment. 'n fa!t, assertions !an in!&%e any !on%itiona statement e earn abo&t those in the net !hapter.
112
Chapter : riting o&r on f&n!tions
For eampe, e -no that if e r&n o&r get_at_content f&n!tion on the D;4 se<&en!e >4$RC> e sho&% get an anser of 0.. $his assertion i test hether thats the !ase: assert get/at/content(*A)%&*# BB 0.
;oti!e the to e<&as signs 5 e earn the reason behin% that in the net !hapter. $he ay that assertion statements or- is very simpe if an assertion t&rns o&t to be fase i.e. if #ython ee!&tes o&r f&n!tion on the inp&t >4$RC> an% the anser isnt 0.= then the program i stop an% e i get an !ssertionError . 4ssertions are &sef& in a n&mber of ays. $hey provi%e a means for &s to !he! hether o&r f&n!tions are or-ing as inten%e% an% therefore hep &s tra!- %on errors in o&r programs. 'f e get some &nepe!te% o&tp&t from a program that &ses a parti!&ar f&n!tion, an% the assertion tests for that f&n!tion a pass, then e !an be !onfi%ent that the error %oesnt ie in the f&n!tion b&t in the !o%e that !as it. $hey aso et &s mo%ify a f&n!tion an% !he!- that e havent intro%&!e% any errors. 'f e have a f&n!tion that passes a series of assertion tests, an% e ma-e some !hanges to it, e !an rer&n the assertion tests an%, ass&ming they a pass, be !onfi%ent that e havent bro-en the f&n!tion 1. 4ssertions are aso &sef& as a form of %o!&mentation. (y in!&%ing a !oe!tion of assertion tests aongsi%e a f&n!tion, e !an sho ea!ty hat o&tp&t is epe!te% from a given inp&t. Finay, e !an &se assertions to test the behavio&r of o&r f&n!tion for &n&s&a inp&ts. For eampe, hat is the epe!te% behavio&r of get_at_content hen given a D;4 se<&en!e that in!&%es &n-non bases &s&ay represente% as N=E 4 sensibe ay to han%e &n-non bases o&% be to e!&%e them from the 4$ !ontent !a!&ation 5 in other or%s, the 4$ !ontent for a given se<&en!e sho&%nt 1
$his i%ea is very simiar to a pro!ess in softare %eveopment !ae% regression testing .
113
Chapter : riting o&r on f&n!tions
be affe!te% by a%%ing a b&n!h of &n-non bases. e !an rite an assertion that epresses this: assert get/at/content(*A)%&<<<<<<<<<<*# BB 0.
$his assertions fais for the !&rrent version of get_at_content . 7oever, e !an easiy mo%ify the f&n!tion to remove a N !hara!ters before !arrying o&t the !a!&ation: def get/at/content(dna, sig/figsB2#! dna B dna.replace(<, # length B len(dna# a/count B dna.upper(#.count(A# t/count B dna.upper(#.count()# at/content B (a/count C t/count# length return round(at/content, sig/figs#
an% no the assertion passes. 'ts !ommon to gro&p a !oe!tion of assertions for a parti!&ar f&n!tion together to test for the !orre!t behavio&r on %ifferent types of inp&t. 7eres an eampe for get_at_content hi!h shos a range of %ifferent types of behavio&r: assert assert assert assert assert assert
get/at/content(*A*# BB 1 get/at/content(*%*# BB 0 get/at/content(*A)%&*# BB 0. get/at/content(*A%%*# BB 0.33 get/at/content(*A%%*, 1# BB 0.3 get/at/content(*A%%*, # BB 0.33333
(ecap 'n this !hapter, eve seen ho pa!-aging &p !o%e into f&n!tions heps &s to manage the !ompeity of arge programs an% promote !o%e re&se. e earne% ho
11B
Chapter : riting o&r on f&n!tions
to %efine an% !a o&r on f&n!tions aong ith vario&s ne ays to s&ppy arg&ments to f&n!tions. e aso oo-e% at a !o&pe of things that are possibe in #ython, b&t rarey a%visabe 5 riting f&n!tions itho&t arg&ments or ret&rn va&es. Finay, e epore% the &se of assertions to test o&r f&n!tions, an% %is!&sse% ho e !an &se them to !at!h errors before they be!ome a probem. $he remaining !hapters in this boo- i ma-e &se of f&n!tions in both the eampes an% the eer!ise so&tions, so ma-e s&re yo& are !omfortabe ith the ne i%eas from this !hapter before moving on.
11
Chapter : riting o&r on f&n!tions
!ercises Percentage of amino acid residues) part one rite a f&n!tion that ta-es to arg&ments 5 a protein se<&en!e an% an amino a!i% resi%&e !o%e 5 an% ret&rns the per!entage of the protein that the amino a!i% ma-es &p. *se the fooing assertions to test yo&r f&n!tion: assert assert assert assert
my/function(*SQS???Q??????PP?P*, my/function(*SQS???Q??????PP?P*, my/function(*SQS???Q??????PP?P*, my/function(*SQS???Q??????PP?P*,
**# *r*# *?*# *R*#
BB BB BB BB
10 0 0
Reminder: if yo&re &sing #ython 2 rather than #ython 3, in!&%e this ine at the top of yo&r program: from //future// import diision
Percentage of amino acid residues) part to Mo%ify the f&n!tion from part one so that it a!!epts a ist of amino a!i% resi%&es rather than a singe one. 'f no ist is given, the f&n!tion sho&% ret&rn the per!entage of hy%rophobi! amino a!i% resi%&es 4, ', +, M, F, , an% S=. o&r f&n!tion sho&% pass the fooing assertions: assert assert assert assert
my/function(*SQS???Q??????PP?P*, my/function(*SQS???Q??????PP?P*, my/function(*SQS???Q??????PP?P*, my/function(*SQS???Q??????PP?P*#
$**# BB $, ?# BB $, S, ?# BB 70 BB "
11G
Chapter : riting o&r on f&n!tions
&olutions Percentage of amino acid residues) part one $his is a simiar probem to ones that eve ta!-e% before, b&t e have to pay attention to the %etais. +ets start ith a pie!e of !o%e that %oes the !a!&ation for a spe!ifi! protein se<&en!e an% amino a!i% !o%e, an% then t&rn it into a f&n!tion. Ca!&ating the per!entage is very simiar to !a!&ating the 4$ !ontent, b&t e i nee% to m&tipe the res&t by 100 to get a per!entage rather than a fra!tion: protein B *SQS???Q??????PP?P* aa B *Q* aa/count B protein.count(aa# protein/length B len(protein# percentage B aa/count K 100 protein/length print(percentage#
;o e ma-e this !o%e into a f&n!tion by t&rning the to variabes protein an% aa into arg&ments, an% ret&rning the per!entage rather than printing it. e a%% in the assertions at the en% of the program to test if the f&n!tion is %oing its 9ob: def get/aa/percentage(protein, aa#! aa/count B protein.count(aa# protein/length B len(protein# percentage B aa/count K 100 protein/length return percentage 4 test assert assert assert assert
the function ith assertions get/aa/percentage(*SQS???Q??????PP?P*, get/aa/percentage(*SQS???Q??????PP?P*, get/aa/percentage(*msrslllrfllfllllpplp*, get/aa/percentage(*SQS???Q??????PP?P*,
**# *r*# *?*# *R*#
BB BB BB BB
10 0 0
11H
Chapter : riting o&r on f&n!tions
&nning the !o%e shos that one of the assertions is faiing 5 the error message tes &s hi!h assertion is the faie% one: assert get/aa/percentage(*SQS???Q??????PP?P*, *r*# BB 10 Assertion9rror
?&r f&n!tion fais to or- hen the protein se<&en!e is in &pper !ase, b&t the amino a!i% resi%&e !o%e is in oer !ase. +oo-ing at the assertions, e !an ma-e an e%&!ate% g&ess that the net one ith the protein in oer !ase an% the amino a!i% in &pper !ase= is probaby going to fai as e. +ets try to fi both of these probems by !onverting both the protein an% the amino a!i% string to &pper !ase at the start of the f&n!tion. e &se the same tri!- as e %i% before of !onverting a string to &pper !ase an% then storing the res&t ba!- in the same variabe: def get/aa/percentage(protein, aa#! 4 conert -oth inputs to upper case protein B protein.upper(# aa B aa.upper(# aa/count B protein.count(aa# protein/length B len(protein# percentage B aa/count K 100 protein/length return percentage
;o a the assertions pass itho&t error.
Percentage of amino acid residues) part to $his eer!ise invoves something that eve not seen before: a f&n!tion that ta-es a ist as one of its arg&ments. 4s in the previo&s eer!ise, e pi!- one of the assertion !ases an% rite the !o%e to sove it first, then t&rn the !o%e into a f&n!tion.
11I
Chapter : riting o&r on f&n!tions
$here are a!t&ay to ays to approa!h this probem. e !an &se a oop to go thro&gh ea!h of the given amino a!i% resi%&es in t&rn, !o&nting &p the n&mber of times they o!!&r in the protein se<&en!e, to get a tota !o&nt. ?r, e !an treat the protein se<&en!e string as a ist as %es!ribe% in the previo&s !hapter= an% as-, for ea!h position, hether the !hara!ter at that position is a member of the ist of amino a!i% resi%&es. e &se the first metho% here in the net !hapter e earn abo&t the toos ne!essary to impement the se!on%. e nee% some ay to -eep a r&nning tota of mat!hing amino a!i%s as e go ro&n% the oop, so e !reate a ne variabe o&tsi%e the oop an% &p%ate it ea!h time ro&n%. $he !o%e insi%e the oop i be <&ite simiar to that from the previo&s eer!ise. 7eres the !o%e ith some print statements so e !an see ea!ty hat is happening: protein B *SQS???Q??????PP?P* aa/list B $, ?, 4 the total aria-le ill hold the total num-er of matching residues total B 0 for aa in aa/list! print(*counting num-er of * C aa# aa B aa.upper(# aa/count B protein.count(aa# 4 add the num-er for this residue to the total count total B total C aa/count print(*running total is * C str(total## percentage B total K 100 len(protein# print(*final percentage is * C str(percentage##
hen e r&n the !o%e, e !an see ho the r&nning tota in!reases ea!h time ro&n% the oop:
11@
Chapter : riting o&r on f&n!tions
counting num-er of running total is 1 counting num-er of ? running total is 11 counting num-er of running total is 13 final percentage is ".0
;o ets ta-e the !o%e an%, 9&st i-e before, t&rn the protein string an% the amino a!i% ist into arg&ments to !reate a f&n!tion: def get/aa/percentage(protein, aa/list#! protein B protein.upper(# protein/length B len(protein# total B 0 for aa in aa/list! aa B aa.upper(# aa/count B protein.count(aa# total B total C aa/count percentage B total K 100 protein/length return percentage
$his f&n!tion passes a the assertion tests e!ept the ast one, hi!h tests the behavio&r hen r&n ith ony one arg&ment. 'n fa!t, #ython never even gets as far as testing the res&t from r&nning the f&n!tion, as e get an error in%i!ating that the f&n!tion %i%nt !ompete: )ype9rror! get/aa/percentage(# ta:es exactly 2 arguments (1 gien#
Fiing the error ta-es ony one !hange: e a%% a %efa&t va&e for aa_list in the first ine of the f&n!tion %efinition:
120
Chapter : riting o&r on f&n!tions
def get/aa/percentage(protein, aa/listB$A,L,?,,,=,R,#! protein B protein.upper(# protein/length B len(protein# total B 0 for aa in aa/list! aa B aa.upper(# aa/count B protein.count(aa# total B total C aa/count percentage B total K 100 protein/length return percentage
an% no a the assertions pass.
121
&:
Chapter G: Con%itiona tests
Conditional tests
Programs nee" to make "ecisions 'f e oo- ba!- at the eampes an% eer!ises in previo&s !hapters, something that stan%s o&t is the a!- of %e!isionma-ing. eve gone from %oing simpe !a!&ations on in%ivi%&a bits of %ata to !arrying o&t more !ompi!ate% pro!e%&res on !oe!tions of %ata, b&t the ay that ea!h bit of %ata a se<&en!e, a base, a spe!ies name, an eon= has been treate% i%enti!ay. eaife probems, hoever, often re<&ire o&r programs to a!t as %e!isionma-ers to eamine a property of some bit of %ata an% %e!i%e hat to %o ith it. 'n this !hapter, e see ho to %o that &sing con"itional statements. Con%itiona statements are feat&res of #ython that ao &s to b&i% %e!ision points in o&r !o%e. $hey ao o&r programs to %e!i%e hi!h o&t of a n&mber of possibe !o&rses of a!tion to ta-e 5 instr&!tions i-e > print the name of the se;uence if it7s longer than )33 bases> or > group two samples together if they were collecte" less than 13 metres apart >. (efore e !an start &sing !on%itiona statements, hoever, e nee% to &n%erstan% con"itions.
4on"itions0 'rue an" /alse 4 con"ition is simpy a bit of !o%e that !an pro%&!e a tr&e or fase anser. $he easiest ay to &n%erstan% ho !on%itions or- in #ython is try o&t a fe eampes. $he fooing eampe prints o&t the res&t of testing or evaluating = a b&n!h of %ifferent !on%itions 5 some mathemati!a eampes, some &sing string metho%s, an% one for testing if a va&e is in!&%e% in a ist:
122
1 2 3 4 5 6 7 8 9 10
Chapter G: Con%itiona tests
print(3 BB # print(3 + # print(3 ;B# print(len(*A)%&*# + # print(*%AA))&*.count(*)*# + 1# print(*A)%&))*.startsith(*A)%*## print(*A)%&))*.endsith(*)))*## print(*A)%&))*.isupper(## print(*A)%&))*.isloer(## print(** in $**, *=*, *?*#
'f e oo- at the o&tp&t, e !an see &se the ine n&mbers to mat!h &p ea!h !on%ition ith its res&t: 1 2 3 4 5 6 7 8 9 10
alse alse )rue alse )rue )rue alse )rue alse )rue
(&t hats a!t&ay being printe% hereE 4t first gan!e, it oo-s i-e ere printing the strings >$r&e> an% >Fase>, b&t those strings %ont appear anyhere in o&r !o%e. hat is a!t&ay being printe% is the spe!ia b&itin va&es that #ython &ses to represent tr&e an% fase 5 they are !apitaie% so that e -no theyre these spe!ia va&es. e !an sho that these va&es are spe!ia by trying to print them. $he fooing !o%e r&ns itho&t errors note the absen!e of <&otation mar-s=: print()rue# print(alse#
123
Chapter G: Con%itiona tests
hereas trying to print arbitrary &n<&ote% or%s: print(ello#
!a&ses a NameError. $heres a i%e range of things that e !an in!&%e in !on%itions, an% it o&% be impossibe to give an eha&stive ist here. $he basi! b&i%ing bo!-s are: •
e<&as represente% by @@=
•
greater an% ess than represente% by an% H=
•
greater an% ess than or e<&a to represente% by @ an% H@=
•
not e<&a represente% by I@=
•
is a va&e in a ist represente% by in=
•
are to ob9e!ts the same1 represente% by is=
Many %ata types aso provi%e metho%s that ret&rn $r&e or Fase va&es, hi!h are often a ot more !onvenient to &se than the b&i%ing bo!-s above. eve area%y seen a fe in the !o%e sampe above: for eampe, strings have a startsith metho% that ret&rns tr&e if the string starts ith the string given as an arg&ment. e mention these tr&e/fase metho%s hen they !ome &p. ;oti!e that the test for e<&aity is to e<&as signs, not one. Forgetting the se!on% e<&as sign i !a&se an error. ;o that e -no ho to epress tests as !on%itions, ets see hat e !an %o ith them.
1
4 %is!&ssion of hat this a!t&ay means in #ython is beyon% the s!ope of this boo-, so e avoi% &sing this !omparison for the !hapter.
12B
Chapter G: Con%itiona tests
if statements $he simpest -in% of !on%itiona statement is an if statement. 7opef&y the synta is fairy simpe to &n%erstan%: expression/leel B 12 if expression/leel + 100! print(*gene is highly expressed*#
e rite the or% if, fooe% by a !on%ition, an% en% the first ine ith a !oon. $here foos a bo!- of in%ente% ines of !o%e the bo"y of the if statement=, hi!h i ony be ee!&te% if the !on%ition is tr&e. $his !oonp&sbo!- pattern sho&% be famiiar to yo& from the !hapters on oops an% f&n!tions. Most of the time, e ant to &se an if statement to test a property of some variabe hose va&e e %ont -no at the time hen e are riting the program. $he eampe above is obvio&sy &seess, as the va&e of the expression_leel variabe is not going to !hange8 7eres a sighty more interesting eampe: e %efine a ist of gene a!!ession names an% print o&t 9&st the ones that start ith >a>: accs B $a-", -hG', h7", ayJ3, apJ7, -d72 for accession in accs! if accession.startsith(a#! print(accession#
+oo-ing at the o&tp&t aos &s to !he!- that this or-s as inten%e%: a-" ayJ3 apJ7
12
Chapter G: Con%itiona tests
'f yo& ta-e a !ose oo- at the !o%e above, yo& see something interesting 5 the ines of !o%e insi%e the oop are in%ente% 9&st as eve seen before=, b&t the ine of !o%e insi%e the if statement is in%ente% tice 5 on!e for the oop, an% on!e for the if. $his is the first time eve seen m&tipe eves of in%entation, b&t its very !ommon on!e e start or-ing ith arger programs 5 henever e have one oop or if statement neste% insi%e another, e have this type of in%entation. #ython is <&ite happy to have as many eves of in%entation as nee%e%, b&t yo& nee% to -eep !aref& tra!- of hi!h ines of !o%e beong at hi!h eve. 'f yo& fin% yo&rsef riting a pie!e of !o%e that re<&ires more than three eves of in%entation, its generay an in%i!ation that that pie!e of !o%e sho&% be t&rne% into a f&n!tion.
else statements Cosey reate% to the if statement is the else statement. $he eampes above &se a yes9no type of %e!isionma-ing: sho&% e print the gene a!!ession n&mber or notE ?ften e nee% an either9or type of %e!ision, here e have to possibe a!tions to ta-e. $o %o this, e !an a%% on an else !a&se after the en% of the bo%y of an if statement: expression/leel B 12 if expression/leel + 100! print(*gene is highly expressed*# else! print(*gene is loly expressed*#
$he else statement %oesnt have any !on%ition of its on 5 rather, the ese statement bo%y is ee!&te hen the if statement to hi!h its atta!he% is not ee!&te%. 7eres an eampe hi!h &ses if an% else to spit &p a ist of a!!ession names into to %ifferent fies 5 a!!essions that start ith >a> go into the first fie, an% a other a!!essions go into the se!on% fie:
12G
Chapter G: Con%itiona tests
file1 B open(*one.txt*, **# file2 B open(*to.txt*, **# accs B $a-", -hG', h7", ayJ3, apJ7, -d72 for accession in accs! if accession.startsith(a#! file1.rite(accession C *n*# else! file2.rite(accession C *n*#
;oti!e ho there are m&tipe in%entation eves as before, b&t that the if an% else statements are at the same eve.
elif statements hat if e have more than to possibe bran!hesE For eampe, say e ant three fies of a!!ession names: ones that start ith >a>, ones that start ith >b>, an% a others. e !o&% have a se!on% if statement neste% insi%e the else !a&se of the first if statement: file1 B open(*one.txt*, **# file2 B open(*to.txt*, **# file3 B open(*three.txt*, **# accs B $a-", -hG', h7", ayJ3, apJ7, -d72 for accession in accs! if accession.startsith(a#! file1.rite(accession C *n*# else! if accession.startsith(-#! file2.rite(accession C *n*# else! file3.rite(accession C *n*#
$his or-s, b&t is %iffi!&t to rea% 5 e !an <&i!-y see that e nee% an etra eve of in%entation for every a%%itiona !hoi!e e ant to in!&%e. $o get ro&n% this,
12H
Chapter G: Con%itiona tests
#ython has an elif statement, hi!h merges together else an% if an% aos &s to rerite the above eampe in a m&!h more eegant ay: file1 B open(*one.txt*, **# file2 B open(*to.txt*, **# file3 B open(*three.txt*, **# accs B $a-", -hG', h7", ayJ3, apJ7, -d72 for accession in accs! if accession.startsith(a#! file1.rite(accession C *n*# elif accession.startsith(-#! file2.rite(accession C *n*# else! file3.rite(accession C *n*#
;oti!e ho this version of the !o%e ony nee%s to eves of in%ention. 'n fa!t, &sing elif e !an have any n&mber of bran!hes an% sti ony re<&ire a singe etra eve of in%entation: for accession in accs! if accession.startsith(a#! file1.rite(accession C *n*# elif accession.startsith(-#! file2.rite(accession C *n*# elif accession.startsith(c#! file3.rite(accession C *n*# elif accession.startsith(d#! file'.rite(accession C *n*# elif accession.startsith(e#! file.rite(accession C *n*# else! file".rite(accession C *n*#
12I
Chapter G: Con%itiona tests
while loops 7eres 7eres one fina thing e !an %o ith !on%itions: &se them to %etermine hen to eit a oop. 'n !hapter B e earne% abo&t oops that iterate over a !oe!tion of items i-e a ist, a string or a fie=. #ython has another type of oop !ae% a hile oop. ather than r&nning a set n&mber of times, a hile oop r&ns &nti some !on%ition is met. For eampe, heres heres a bit of !o%e that in!rements a !o&nt variabe by one ea!h time ro&n% the oop, stopping hen the !o&nt variabe rea!hes ten: count B 0 hile count;10! print(count# count B count C 1
(e!a&se norma oops in #ython are so poerf&1, hile oops are &se% m&!h ess fre<&enty than in other ang&ages, so e ont %is!&ss them f&rther.
8uil"ing up comple! con"itions hat if e ante% to epress a !on%ition that as ma%e &p of severa partsE 'magine e ant to go thro&gh o&r ist of a!!essions an% print o&t ony the ones that start ith >a> an% en% ith >3>. >3>. e !o&% &se to neste% if statements: statem ents: accs B $a-", -hG', h7", ayJ3, apJ7, -d72 for accession in accs! if accession.startsith(a#! if accession.endsith(3#! print(accession#
b&t this brings in an etra, &nnee%e% eve of in%ention. 4 better ay is to 9oin &p the to !on%ition ith and to ma-e a !ompe epression:
1
).g. the eampe eampe !o%e !o%e here here !o&% !o&% be bette betterr a!!om a!!ompish pishe% e% ith ith a range.
12@
Chapter G: Con%itiona tests
accs B $a-", -hG', h7", ayJ3, apJ7, -d72 for accession in accs! if accession.startsith(a# and accession.endsith(3#! print(accession#
$his version is ni!er in to ays: it %oesnt re<&ire the etra eve of in%entation, an% the !on%ition rea%s in a very very nat&ra nat&ra ay. e !an aso &se or to 9oin &p to !on%itions, to pro%&!e a !ompe !on%ition that i be tr&e if either of the to simpe !on%itions are tr&e: accs B $a-", -hG', h7", ayJ3, apJ7, -d72 for accession in accs! if accession.startsith(a# or accession.startsith(-#! print(accession#
e !an even 9oin &p !ompe !on%itions to ma-e more !ompe !on%itions 5 heres an eampe hi!h prints a!!essions if they start ith either >a> or >b> an% en% ith >B>: accs B $a-", -hG', h7", ayJ3, apJ7, -d72 for acc in accs! if (acc.startsith(a# or acc.startsith(-## and acc.endsith('#! print(acc#
;oti!e ho e have to in!&%e parentheses in the above eampe to avoi% ambig&ity. Finay, e !an negate any type of !on%ition by prefiing it ith the or% not. $his eampe i print o&t a!!essions that start ith >a> an% don-t en% ith G: accs B $a-", -hG', h7", ayJ3, apJ7, -d72 for acc in accs! if acc.startsith(a# and not acc.endsith("#! print(acc#
130
Chapter G: Con%itiona tests
(y &sing a !ombination of and, or an% not aong ith parentheses here ne!essary= e !an b&i% &p arbitrariy !ompe !on%itions. $hese three or%s are !oe!tivey -non as boolean operators an% !rop &p in a ot of pa!es. For eampe, if yo& ante% to sear!h for information on &sing #ython in bioogy, bioogy, b&t %i%nt ant to see pages that ta-e% abo&t bioogy of sna-es, yo& yo& might %o a sear!h for >biology python Bsnake>. $his is a!t&ay a !ompe !on%ition 9&st i-e the ones above 5 Rooge a&tomati!ay a%%s an" beteen beteen or%s, an% &ses the hyphen to mean not . "o yo&re yo&re as-ing for pages that mention python an" bioogy bioogy b&t not sna-es. sna-es.
Writing true9false functions "ometimes e ant to rite a f&n!tion that !an be &se% in a !on%ition. $his is very easy to %o 5 e e 9&st ma-e s&re that o&r f&n!tion aays ret&rns either $r&e or Fase. emember that $r&e an% Fase are b&itin va&es in #ython, so they !an be passe% aro&n%, store% in variabes, an% ret&rne%, 9&st i-e n&mbers or strings. 7eres a f&n!tion that %etermines hether or not a D;4 se<&en!e is 4$ri!h e say that a se<&en!e is 4$ri!h if it has an 4$ !ontent of more than 0.G=: def is/at/rich(dna#! length B len(dna# a/count B dna.upper(#.count(A# t/count B dna.upper(#.count()# at/content B (a/count C t/count# length if at/content + 0."! return )rue else! return alse
e test this f&n!tion on a fe se<&en!es to see if it or-s:
131
Chapter G: Con%itiona tests
print(is/at/rich(*A))A)&)A&)A*## print(is/at/rich(*&%%&A%&%&)*##
$he o&tp&t shos that the f&n!tion ret&rns $r&e or Fase 9&st i-e the other !on%itions eve been oo-ing at: )rue alse
$herefore e !an &se o&r f&n!tion in an if statement: if is/at/rich(my/dna#! 4 do something ith the seuence
(e!a&se the ast fo&r ines of o&r f&n!tion are %evote% to eva&ating a !on%ition an% ret&rning $r&e or Fase, e !an rite a sighty more !ompa!t version. 'n this eampe e eva&ate the !on%ition, an% then ret&rn the res&t right aay: def is/at/rich(dna#! length B len(dna# a/count B dna.upper(#.count(A# t/count B dna.upper(#.count()# at/content B (a/count C t/count# length return at/content + 0."
$his is a itte more !on!ise, an% aso easier to rea% on!e yo&re famiiar ith the i%iom.
(ecap 'n this short !hapter, eve %eat ith to things: !on%itions, an% the statements that &se them.
132
Chapter G: Con%itiona tests
eve seen ho simpe !on%itions !an be 9oine% together to ma-e more !ompe ones, an% ho the !on!epts of tr&th an% fasehoo% are b&it in to #ython on a f&n%amenta eve. eve aso seen ho e !an in!orporate $r&e an% Fase in o&r on f&n!tions in a ay that aos them to be &se% as part of !on%itions. eve been intro%&!e% to fo&r %ifferent toos that &se !on%itions 5 if, else, elif, an% hile 5 in approimate or%er of &sef&ness. o& probaby fin%, in the programs that yo& rite an% in yo&r so&tions to the eer!ises in this boo-, that yo& &se if an% else very fre<&enty, elif o!!asionay, an% hile amost never.
133
Chapter G: Con%itiona tests
!ercises 'n the chapter- fo%er in the eer!ises %onoa%, yo& fin% a tet fie !ae% "ata=csv , !ontaining some ma%e&p %ata for a n&mber of genes. )a!h ine !ontains the fooing fie%s for a singe gene in this or%er: spe!ies name, se<&en!e, gene name, epression eve. $he fie%s are separate% by !ommas hen!e the name of the fie 5 !sv stan%s for Comma "eparate% Sa&es=. $hin- of it as a representation of a tabe in a sprea%sheet 5 ea!h ine is a ro, an% ea!h fie% in a ine is a !o&mn. 4 the eer!ises for this !hapter &se the %ata rea% from this fie. Reminder: if yo&re &sing #ython 2 rather than #ython 3, in!&%e this ine at the top of yo&r programs: from //future// import diision
everal species #rint o&t the gene names for a genes beonging to rosophila melanogaster or rosophila simulans.
#engt+ range #rint o&t the gene names for a genes beteen @0 an% 110 bases ong.
4T content #rint o&t the gene names for a genes hose 4$ !ontent is ess than 0. an% hose epression eve is greater than 200.
13B
Chapter G: Con%itiona tests
Complex condition #rint o&t the gene names for a genes hose name begins ith >-> or >h> e!ept those beonging to rosophila melanogaster .
7ig+ lo medium For ea!h gene, print o&t a message giving the gene name an% saying hether its 4$ !ontent is high greater than 0.G=, o ess than 0.B= or me%i&m beteen 0.B an% 0.G=.
13
Chapter G: Con%itiona tests
&olutions everal species $hese eer!ises are somehat more !ompi!ate% than previo&s ones, an% theyre going to re<&ire materia from m&tipe %ifferent !hapters to sove. $he first probem is to %ea ith the format of the %ata fie. ?pen it &p in a tet e%itor an% ta-e a oo- before !ontin&ing. e -no that ere going to have to open the fie !hapter 3= an% pro!ess the !ontents inebyine !hapter B=. $o %ea ith ea!h ine, e have to spit it to ma-e a ist of !o&mns !hapter B=, then appy the !on%ition this !hapter= in or%er to fig&re o&t hether or not e sho&% print it. 7eres a program that i rea% ea!h ine from the fie, spit it &sing !ommas as the %eimiter, then assign ea!h of the fo&r !o&mns to a variabe an% print the gene name: data B open(*data.cs*# for line in data! columns B line.rstrip(*n*#.split(*,*# species B columns$0 seuence B columns$1 name B columns$2 expression B columns$3 print(name#
;oti!e that e &se rstrip to remove the neine from the en% of the !&rrent ine before spitting it. e -no the or%er of the fie%s in the ine be!a&se they ere mentione% in the eer!ise %es!ription, so e !an easiy assign them to the fo&r variabes. $his program %oesnt %o anything &sef&, b&t e !an !he!- the o&tp&t to !onfirm that it gets the names right:
13G
Chapter G: Con%itiona tests
:dy"'7 Ddg7"" :dy33 hdt73J hdu0' teg'3"
;o e !an a%% in the !on%ition. e ant to print the name if the spe!ies is eit+er rosophila melanogaster or rosophila simulans. 'f the spe!ies name is neit+er of those to, then e %ont ant to %o anything. $his is a yes9no type %e!ision, so e nee% an if statement: data B open(*data.cs*# for line in data! columns B line.rstrip(*n*#.split(*,*# species B columns$0 seuence B columns$1 name B columns$2 expression B columns$3 if species BB *@rosophila melanogaster* or species BB *@rosophila simulans*! print(name#
$he ine !ontaining the if statement is <&ite ong, so it raps aro&n% onto the net ine on this page, b&t its sti 9&st a singe ine in the program fie. e !an !he!- the o&tp&t e get: :dy"'7 Ddg7"" :dy33
against the !ontents of the fie, an% !onfirm that the program is or-ing.
13H
Chapter G: Con%itiona tests
#engt+ range e !an re&se a arge part of the !o%e from the previo&s eer!ise to hep sove this one. e have another !ompe !on%ition: e ony ant to print names for genes hose ength is beteen @0 an% 110 bases 5 in other or%s, genes hose ength is greater than @0 and ess than 110. e have to !a!&ate the ength &sing the len f&n!tion. ?n!e eve %one that the rest of the program is <&ite straightforar%: data B open(*data.cs*# for line in data! columns B line.rstrip(*n*#.split(*,*# species B columns$0 seuence B columns$1 name B columns$2 expression B columns$3 if len(seuence# + J0 and len(seuence# ; 110! print(name#
4T content $his eer!ise has a !ompe !on%ition i-e the others, b&t it aso re<&ires &s to %o a bit more !a!&ation 5 e nee% to be abe to !a!&ate the 4$ !ontent of ea!h se<&en!e. ather than starting from s!rat!h, e simpy &se the f&n!tion that e rote in the previo&s !hapter an% in!&%e it at the start of the program. ?n!e eve %one that, its 9&st a !ase of &sing the o&tp&t from get_at_content as part of the !on%ition:
13I
Chapter G: Con%itiona tests
4 our function to get A) content def get/at/content(dna#! length B len(dna# a/count B dna.upper(#.count(A# t/count B dna.upper(#.count()# at/content B (a/count C t/count# length return at/content data B open(*data.cs*# for line in data! columns B line.rstrip(*n*#.split(*,*# species B columns$0 seuence B columns$1 name B columns$2 expression B columns$3 if get/at/content(seuence# ; 0. and expression + 200! print(name#
Complex condition $here are no !a!&ations to !arry o&t for this eer!ise 5 the !ompeity !omes from the fa!t that there are three !omponents to the !on%ition, an% they have to be 9oine% together in the right ay: data B open(*data.cs*# for line in data! columns B line.rstrip(*n*#.split(*,*# species B columns$0 seuence B columns$1 name B columns$2 expression B columns$3 if (name.startsith(:# or name.startsith(h## and species 5B *@rosophila melanogaster*! print(name#
$he ine !ontaining the if statement is <&ite ong, so it raps aro&n% onto the net ine on this page, b&t its sti 9&st a singe ine in the program fie. $here are to
13@
Chapter G: Con%itiona tests
%ifferent ays to epress the re<&irement that the name is not rosophila melanogaster= 'n the above eampe eve &se% the note<&as sign 8Z= b&t e !o&% aso have &se% the not booean operator: if (name.startsith(:# or name.startsith(h## and not species BB *@rosophila melanogaster*!
7ig+ lo medium ;o e !ome to an eer!ise that re<&ires the &se of m&tipe bran!hes. e have three %ifferent printing options for ea!h gene 5 high, o an% me%i&m 5 so e nee% an if..elif..else se!tion to han%e the !on%itions. e &se the get_at_content f&n!tion as before: 4 our function to get A) content def get/at/content(dna#! length B len(dna# a/count B dna.upper(#.count(A# t/count B dna.upper(#.count()# at/content B (a/count C t/count# length return at/content data B open(*data.cs*# for line in data! columns B line.rstrip(*n*#.split(*,*# species B columns$0 seuence B columns$1 name B columns$2 expression B columns$3 if get/at/content(seuence# + 0."! print(name C * has high A) content*# elif get/at/content(seuence# ; 0.'! print(name C * has lo A) content*# else! print(name C * has medium A) content*#
1B0
Chapter G: Con%itiona tests
Che!-ing the o&tp&t !onfirms that the !on%itions are or-ing: :dy"'7 Ddg7"" :dy33 hdt73J hdu0' teg'3"
has has has has has has
high A) content medium A) content medium A) content lo A) content medium A) content medium A) content
1B1
Chapter H: eg&ar epressions
$:
Regular expressions
'he importance of patterns in biology 4 ot of hat e %o hen riting programs for bioogy !an be %es!ribe% as sear!hing for patterns in strings. $he obvio&s eampes !ome from the anaysis of bioogi!a se<&en!e %ata 5 remember that D;4, ;4 an% protein se<&en!es are 9&st strings. Many of the things e ant to oo- for in bioogi!a se<&en!es !an be %es!ribe% in terms of patterns: •
protein %omains
•
D;4 trans!ription fa!tor bin%ing motifs
•
restri!tion enyme !&t sites
•
%egenerate #C primer sites
•
r&ns of monon&!eoti%es
7oever, its not 9&st se<&en!e %ata that !an have interesting patterns. 4s e %is!&sse% in !hapter 3, most of the other types of %ata e have to %ea ith in bioogy !omes in the form of strings 1 insi%e tet fies 5 things i-e:
1
•
rea% mapping o!ations
•
geographi!a sampe !oor%inates
•
taonomi! names
•
gene names
•
gene a!!ession n&mbers
•
(+4"$ sear!hes
;ote that atho&gh many of the things in this ist are n&meri!a %ata, theyre sti rea% in to #ython programs as strings an% nee% to be manip&ate% as s&!h.
1B2
Chapter H: eg&ar epressions
'n previo&s !hapters, eve oo-e% at some programming tas-s that invove pattern re!ognition in strings. eve seen ho to !o&nt in%ivi%&a amino a!i% resi%&es an% even gro&ps of amino a!i% resi%&es= in protein se<&en!es !hapter =, an% ho to i%entify restri!tion enyme !&t sites in D;4 se<&en!es !hapter 2=. eve aso seen ho to eamine parts of gene names an% mat!h them against in%ivi%&a !hara!ters !hapter G=. $he !ommon theme among a these probems is that they invove sear!hing for a fixed set of !hara!ters. (&t there are many probems that e ant to sove that re<&ire more feibe patterns. For eampe: •
•
•
Riven a D;4 se<&en!e, hats the ength of the poy4 taiE Riven a gene a!!ession name, etra!t the part beteen the thir% !hara!ter an% the &n%ers!ore Riven a protein se<&en!e, %etermine if it !ontains this highyre%&n%ant %omain motif
(e!a&se these types of probems !rop &p in so many %ifferent fie%s, theres a stan%ar% set of toos in #ython 1 for %eaing ith them: regular e!pressions= eg&ar epressions2 are a topi! that might not be !overe% in a generap&rpose programming boo-, b&t be!a&se theyre so &sef& in bioogy, ere going to %evote the hoe of this !hapter to oo-ing at them. 4tho&gh the toos for %eaing ith reg&ar epressions are b&it in to #ython, they are not ma%e a&tomati!ay avaiabe hen yo& rite a program. 'n or%er to &se them e m&st first ta- abo&t mo%&es.
1 2
4n% in many other ang&ages an% &tiities. $he name is often abbreviate% to rege! .
1B3
Chapter H: eg&ar epressions
o"ules in Python $he f&n!tions an% %ata types that eve %is!&sse% so far in this boo- have been ones that are i-ey to be nee%e% in pretty m&!h every program 5 toos for %eaing ith strings an% n&mbers, for rea%ing an% riting fies, an% for manip&ating ists of %ata. 4s s&!h, they are a&tomati!ay ma%e avaiabe hen e start to !reate a #ython program. 'f e ant to open a fie, e simpy rite a statement that &ses the open f&n!tion. 7oever, theres another !ategory of toos in #ython hi!h are more spe!iaie%. eg&ar epressions are one eampe, b&t there is a arge ist of spe!iaie% toos hi!h are very &sef& hen yo& nee% them1, b&t are not i-ey to be nee%e% for the ma9ority of programs. )ampes in!&%e toos for %oing a%van!e% mathemati!a !a!&ations, for %onoa%ing %ata from the eb, for r&nning eterna programs, an% for manip&ating %ate/time information. )a!h !oe!tion of spe!iaie% toos 5 reay 9&st a !oe!tion of spe!iaie% functions an% %ata types 5 is !ae% a mo"ule. For reasons of effi!ien!y, #ython %oesnt a&tomati!ay ma-e these mo%&es avaiabe in ea!h ne program, as it %oes ith the more basi! toos. 'nstea%, e have to epi!ity oa% ea!h mo%&e of spe!iaie% toos that e ant to &se insi%e o&r program. $o oa% a mo%&e e &se the import statement2. For eampe, the mo%&e that %eas ith reg&ar epressions is !ae% re, so if e ant to rite a program that &ses the reg&ar epression toos e m&st in!&%e the ine: import re
at the top of o&r program. hen e then ant to &se one of the toos from a mo%&e, e have to prefi it ith the mo%&e name 3. For eampe, to &se the 1 2 3
'n%ee%, this is one of the great strengths of the #ython ang&age. $his is the reason for the from __future__ import diision statement that e have to in!&%e if ere &sing #ython 2. $here are ays ro&n% this, b&t e ont !onsi%er them in this boo-.
1BB
Chapter H: eg&ar epressions
reg&ar epression search f&n!tion hi!h e %is!&ss ater in this !hapter= e have to rite: re.search(pattern, string#
rather than simpy: search(pattern, string#
'f e forget to import the mo%&e hi!h e ant to &se, or forget to in!&%e the mo%&e name as part of the f&n!tion !a, e i get a NameError . e en!o&nter vario&s other mo%&e in the rest of this boo-. For the rest of this !hapter spe!ifi!ay, a !o%e eampes i re<&ire the import re statement in or%er to or-. For !arity, e ont in!&%e it, so if yo& ant try r&nning any of the !o%e in this !hapter, yo& nee% to a%% it at the top.
(aw strings riting reg&ar epression patterns, as e see in the very net se!tion of this !hapter, re<&ires &s to type a ot of spe!ia !hara!ters. e!a from !hapter 2 that !ertain !ombinations of !hara!ters are interprete% by #ython to have spe!ia meaning. For eampe, \n means start a new line, an% \t means insert a tab character . *nfort&natey, there are a imite% n&mber of spe!ia !hara!ters to go ro&n%, so some of the !hara!ters that have a spe!ia meaning in reg&ar epressions !ash ith the !hara!ters that alread* have a spe!ia meaning. #ythons ay ro&n% this probem is to have a spe!ia r&e for strings: if e p&t the etter r imme%iatey before the opening <&otation mar-, then any spe!ia !hara!ters insi%e the string are ignore%:
1B
Chapter H: eg&ar epressions
print(r*tn*#
$he r stan%s for raw , hi!h is #ythons %es!ription for a string here spe!ia !hara!ters are ignore%. ;oti!e that the r goes outside the <&otation mar-s 5 it is not part of the string itsef. e !an see from the o&tp&t that the above !o%e prints o&t the string 9&st as eve ritten it: tn
itho&t any tabs or ne ines. o& see this spe!ia raw notation &se% in a the reg&ar epression !o%e eampes in this !hapter.
&earching for a pattern in a string e start off ith the simpest reg&ar epression too. re.search is a tr&e/fase f&n!tion that %etermines hether or not a pattern appears somehere in a string. 't ta-es to arg&ments, both strings. $he first arg&ment is the pattern that yo& ant to sear!h for, an% the se!on% arg&ment is the string that yo& ant to sear!h in. For eampe, heres ho e test if a D;4 se<&en!e !ontains an )!o' restri!tion site: dna B *A)&%&%AA))&A&* if re.search(r*%AA))&*, dna#! print(*restriction site found5*#
;oti!e that eve &se% the ra notation for the pattern, even tho&gh its not stri!ty ne!essary as it %oesnt !ontain any spe!ia !hara!ters.
4lternation $he above eampe isnt parti!&ary interesting, as the restri!tion motif has no variation. +ets try it ith the 4va'' motif, hi!h !&ts at to %ifferent motifs:
1BG
Chapter H: eg&ar epressions
RR4CC an% RR$CC. e !an &se the te!hni<&es e earne% in the previo&s !hapter to ma-e a !ompe !on%ition &sing or: dna B *A)&%&%AA))&A&* if re.search(r*%%A&&*, dna# or re.search(r*%%)&&*, dna#! print(*restriction site found5*#
(&t a better ay is to !apt&re the variation in the 4va'' site &sing a reg&ar epression: dna B *A)&%&%AA))&A&* if re.search(r*%%(AT)#&&*, dna#! print(*restriction site found5*#
7ere ere &sing the aternation feat&re of reg&ar epressions. 'nsi%e parentheses, e rite the aternatives separate% by a pipe !hara!ter, so +!J", means either A or ' . $his ets &s rite a singe pattern 5 ##+!J",CC 5 hi!h !apt&res the variation in the motif.
C+aracter groups $he (is' restri!tion enyme !&ts at an even i%er range of motifs 5 the pattern is RC;RC, here ; represents any base. e !an &se the same aternation te!hni<&e to sear!h for this pattern: dna B *A)&%&%AA))&A&* if re.search(r*%&(AT)T%T&#%&*, dna#! print(*restriction site found5*#
7oever, theres another reg&ar epression feat&re that ets &s rite the pattern more !on!isey. 4 pair of s<&are bra!-ets ith a ist of !hara!ters insi%e them !an represent any one of these !hara!ters. "o the pattern #C!"#C2#C i mat!h
1BH
Chapter H: eg&ar epressions
#C!#C, #C"#C, #C##C an% #CC#C. 7eres the same program &sing !hara!ter gro&ps: dna B *A)&%&%AA))&A&* if re.search(r*%&$A)%&%&*, dna#! print(*restriction site found5*#
'f e ant a !hara!ter in a pattern to mat!h an* !hara!ter in the inp&t, e !an &se a perio% 5 the pattern #C.#C o&% mat!h a fo&r possibiities. 7oever, the perio% o&% aso mat!h any !hara!ter hi!h is not a D;4 base, or even a etter. $herefore, the hoe pattern o&% aso mat!h #C7#C, #CK#C an% #C?#C, hi!h may not be hat e ant. "ometimes its easier, rather than isting a the a!!eptabe !hara!ters, to spe!ify the !hara!ters that e don-t ant to mat!h. #&tting a !aret U at the start of a !hara!ter gro&p i-e this UXN i negate it, an% mat!h any !hara!ter that isn-t in the gro&p.
uantifiers $he reg&ar epression feat&res %is!&sse% above et &s %es!ribe variation in the in%ivi%&a !hara!ters of patterns. 4nother gro&p of feat&res, ;uantifiers, et &s %es!ribe variation in the n&mber of times a se!tion of a pattern is repeate%. 4 <&estion mar- imme%iatey fooing a !hara!ter means that that !hara!ter is optiona 5 it !an mat!h either ;ero or one times. "o in the pattern #!"LC the " is optiona, an% the pattern i mat!h either #!"C or #!C. 'f e ant to appy a <&estion mar- to more than one !hara!ter, e !an gro&p the !hara!ters in parentheses. For eampe, in the pattern RRR444=E$$$ the gro&p of three !s is optiona, so the pattern i mat!h either ###!!!""" or ###""". 4 p&s sign imme%iatey fooing a !hara!ter or gro&p means that the !hara!ter or gro&p must be present b&t !an be repeate% any n&mber of times 5 in other or%s,
1BI
Chapter H: eg&ar epressions
it i mat!h one or more times. For eampe, the pattern ###!F""" i mat!h three #s, fooe% by one or more !s, fooe% by three "s. "o it i mat!h ###!""", ###!!"", ###!!!"" , et!. b&t not ###""". 4n asteris- imme%iatey fooing a !hara!ter or gro&p means that the !hara!ter or gro&p is optiona, b&t !an aso be repeate%. 'n other or%s, it i mat!h ;ero or more times. For eampe, the pattern ###!4""" i mat!h three #s, fooe% by ero or more !s, fooe% by three "s. "o it i mat!h ###""", ###!""", ###!!""" , et!. 'f e ant to spe!ify a spe!ifi! n&mber of repeats, e !an &se !&ry bra!-ets. Fooing a !hara!ter or gro&p ith a single n&mber insi%e !&ry bra!-ets i mat!h ea!ty that n&mber of repeats. For eampe, the pattern #!M1" i mat!h #!!!!!" b&t not #!!!!" or #!!!!!!" . Fooing a !hara!ter or gro&p ith a pair of numbers insi%e !&ry bra!-ets separate% ith a !omma aos &s to spe!ify an a!!eptabe range of n&mber of repeats. For eampe, the pattern #!M6%8" i mat!h #!!", #!!!" an% #!!!!" b&t not #!" or #!!!!!".
Positions $he fina set of reg&ar epression toos ere going to oo- at %ont represent !hara!ters at a, b&t rather positions in the inp&t string. $he !aret symbo O mat!hes the start of a string, an% the %oar symbo mat!hes the end of a string. $he pattern O!!! i mat!h !!!""" b&t not ###!!!""" . $he pattern ### i mat!h !!!### b&t not !!!###CCC.
1B@
Chapter H: eg&ar epressions
Combining $he rea poer of reg&ar epressions !omes from !ombining these toos. e !an &se <&antifiers together ith aternations an% !hara!ter gro&ps to spe!ify very feibe patterns. For eampe, heres a !ompe pattern to i%entify f&ength e&-aryoti! messenger ;4 se<&en!es: 8A)%$A)%&U30,1000VAU,10V6
ea%ing the pattern from eft to right, it i mat!h: •
an 4$R start !o%on at the beginning of the se<&en!e
•
fooe% by beteen 30 an% 1000 bases hi!h !an be 4, $, R or C
•
fooe% by a poy4 tai of beteen an% 10 bases at the en% of the se<&en!e
4s yo& !an see, reg&ar epressions !an be <&ite tri!-y to rea% &nti yo&re famiiar ith them8 7oever, its e orth investing a bit of time earning to &se them, as the same notation is &se% a!ross m&tipe %ifferent toos. $he reg&ar epression s-is that yo& earn in #ython are transferabe to other programming ang&ages, !omman% ine toos, an% tet e%itors. $he feat&res eve %is!&sse% above are the ones most &sef& in bioogy, an% are s&ffi!ient to ta!-e a the eer!ises at the en% of the !hapter. 7oever, there are many more reg&ar epression feat&res avaiabe in #ython. 'f yo& ant to be!ome a reg&ar epression master, its orth rea%ing &p on gree"y vs= minimal ;uantifiers , backBreferences, lookahea" an% lookbehin" assertions, an% builtBin character classes. (efore e move on to oo- at some more sophisti!ate% &ses of reg&ar epressions, its orth noting that theres a metho% simiar to re.search !ae% re.match . $he %ifferen!e is that re.search i i%entify a pattern o!!&rring an*+ere in the string, hereas re.match i ony i%entify a pattern if it mat!hes the entire string. Most of the time e ant the former behavio&r.
10
Chapter H: eg&ar epressions
!tracting the part of the string that matche" 'n the se!tion above e &se% re.search as the !on%ition in an if statement to %e!i%e hether or not a string !ontaine% a pattern. ?ften in o&r programs, e ant to fin% o&t not ony if a pattern mat!he%, b&t +at part of the string as mat!he%. $o %o this, e nee% to store the res&t of &sing re.search, then &se the group metho% on the res&ting ob9e!t. hen intro%&!ing the re.search f&n!tion above ' rote that it as a tr&e/fase f&n!tion. $hats not e!actly !orre!t tho&gh 5 if it fin%s a mat!h, it %oesnt ret&rn "rue, b&t rather an ob9e!t that is eva&ate% as tr&e in a !on%itiona !ontet 1 if the %istin!tion %oesnt seem important to yo&, then yo& !an safey ignore it=. $he va&e thats a!t&ay ret&rne% is a mat!h ob9e!t 5 a ne %ata type that eve not en!o&ntere% before. +i-e a fie ob9e!t see !hapter 3=, a mat!h ob9e!t %oesnt represent a simpe thing, i-e a n&mber or string. 'nstea%, it represents the res&ts of a reg&ar epression sear!h. 4n% again, 9&st i-e a fie ob9e!t, a mat!h ob9e!t has a n&mber of &sef& metho%s for getting %ata o&t of it. ?ne s&!h metho% is the group metho%. 'f e !a this metho% on the res&t of a reg&ar epression sear!h, e get the portion of the inp&t string that mat!he% the pattern: dna B *A)%A&%)A&%)A&%A&)%* 4 store the match o-Dect in the aria-le m m B re.search(r*%A$A)%&U3VA&*, dna# print(m.group(##
'n the above !o%e, ere sear!hing insi%e a D;4 se<&en!e for #!, fooe% by three bases, fooe% by !C. (y !aing the group metho% on the res&ting mat!h ob9e!t,
1
'f a mat!h isnt fo&n%, then the same thing appies the f&n!tion %oesnt ret&rn 7alse, b&t a %ifferent b&itin va&e 5 None 5 that eva&ates as fase. 'f this %oesnt ma-e sense, %ont orry.
11
Chapter H: eg&ar epressions
e !an see the part of the D;4 se<&en!e that mat!he%, an% fig&re o&t hat the mi%%e three bases ere: %A&%)A&
hat if e ant to etra!t more than one bit of the patternE "ay e ant to mat!h this pattern: %A$A)%&U3VA&$A)%&U2VA&
$hats #!, fooe% by three bases, fooe% by !C, fooe% by to bases, fooe% by !C again. e !an s&rro&n% the bits of the pattern that e ant to etra!t ith parentheses 5 this is !ae% capturing it: %A($A)%&U3V#A&($A)%&U2V#A&
e !an no refer to the !apt&re% bits of the pattern by s&ppying an arg&ment to the group metho%. group+3, i ret&rn the bit of the string mat!he% by the se!tion of the pattern in the first set of parentheses, group+6, i ret&rn the bit mat!he% by the se!on%, et!.: dna B *A)%A&%)A&%)A&%A&)%* 4 store the match o-Dect in the aria-le m m B re.search(r*%A($A)%&U3V#A&($A)%&U2V#A&*, dna# print(*entire match! * C m.group(## print(*first -it! * C m.group(1## print(*second -it! * C m.group(2##
$he o&tp&t shos that the three bases in the first variabe se!tion ere C#", an% the to bases in the se!on% variabe se!tion ere #":
12
Chapter H: eg&ar epressions
entire match! %A&%)A&%)A& first -it! &%) second -it! %)
$etting the position of a match 4s e as !ontaining information abo&t the contents of a mat!h, the mat!h ob9e!t aso ho%s information abo&t the position of the mat!h. $he start an% end metho%s get the positions of the start an% en% of the pattern on the se<&en!e: dna B *A)%A&%)A&%)A&%A&)%* m B re.search(r*%A($A)%&U3V#A&($A)%&U2V#A&*, dna# print(*start! * C str(m.start(### print(*end! * C str(m.end(###
emember that e start !o&nting from ero, so in this !ase, the mat!h starting at the thir% base has a start position of to: start! 2 end! 13
e !an get the start an% en% positions of in%ivi%&a gro&ps by s&ppying a n&mber as the arg&ment to start an% end: dna B *A)%A&%)A&%)A&%A&)%* m B re.search(r*%A($A)%&U3V#A&($A)%&U2V#A&*, dna# print(*start! * C str(m.start(### print(*end! * C str(m.end(### print(*group one start! * C str(m.start(1### print(*group one end! * C str(m.end(1### print(*group to start! * C str(m.start(2### print(*group to end! * C str(m.end(2###
13
Chapter H: eg&ar epressions
'n this parti!&ar !ase, e !o&% fig&re o&t the start an% en% positions of the in%ivi%&a gro&ps from the start an% en% positions of the hoe pattern: start! 2 end! 13 group one group one group to group to
start! ' end! 7 start! J end! 11
b&t that might not aays be possibe for patterns that have variabe ength repeats.
&plitting a string using a regular e!pression ?!!asionay it !an be &sef& to spit a string &sing a reg&ar epression pattern as the %eimiter. $he norma string split metho% %oesnt ao this, b&t the re mo%&e has a split f&n!tion of its on that ta-es a reg&ar epression pattern as an arg&ment. $he first arg&ment is the pattern, the se!on% arg&ment is the string to be spit. 'magine e have a !onsens&s D;4 se<&en!e that !ontains ambig&ity !o%es, an% e ant to etra!t a r&ns of !ontig&o&s &nambig&o&s bases. e nee% to spit the D;4 string herever e see a base that isnt 4, $, R or C: dna B *A&)<%&A)Q%&)A&%)RA&%A)S&%A=)&%* runs B re.split(r*$8A)%&*, dna# print(runs#
e!a that p&tting a !aret U at the start of a !hara!ter gro&p negates it. $he o&tp&t shos ho the f&n!tion or-s 5 the ret&rn va&e is a ist of strings: $A&), %&A), %&)A&%), A&%A), &%A, )&%
1B
Chapter H: eg&ar epressions
/in"ing multiple matches $he eampes eve seen so far %ea ith !ases here ere ony intereste% in a singe o!!&rren!e of a pattern in a string. 'f instea% e ant to fin% every pa!e here a pattern o!!&rs in a string, there are to f&n!tions in the re mo%&e to hep &s.
re.findall ret&rns a ist of a mat!hes of a pattern in a string. $he first arg&ment is the pattern, an% the se!on% arg&ment is the string. "ay e ant to fin% a r&ns of ! an% " in a D;4 se<&en!e onger than five bases: dna B *A&)%&A))A)A)&%)A&%AAA))A)A&%&%&%* runs B re.findall(r*$A)U',100V*, dna# print(runs#
;oti!e that the ret&rn va&e of the findall metho% is not a mat!h ob9e!t 5 it is a straightforar% ist of strings: $A))A)A), AAA))A)A
so e have no ay to etra!t the positions. 'f e ant to %o anything more !ompi!ate% than simpy etra!ting the tet of the mat!hes, e nee% to &se the re.finditer metho%. finditer ret&rns a se<&en!e of mat!h ob9e!ts, so to %o anything &sef& ith it, e nee% to &se the ret&rn va&e in a oop: dna B *A&)%&A))A)A)&%)A&%AAA))A)A&%&%&%* runs B re.finditer(r*$A)U3,100V*, dna# for match in runs! run/start B match.start(# run/end B match.end(# print(*A) rich region from * C str(run/start# C * to * C str(run/end##
1
Chapter H: eg&ar epressions
4s e !an see from the o&tp&t: A) rich region from to 12 A) rich region from 1G to 2"
finditer gives &s !onsi%eraby more feibiity that findall.
(ecap J&st as in the previo&s !hapter, e earne% abo&t to %istin!t !on!epts !on%itions, an% the statements that &se them= in this !hapter e earne% abo&t regular e!pressions, an% the functions that &se them. e starte% ith a brief intro%&!tion to to !on!epts that, hie not part of the reg&ar epression toos, are ne!essary in or%er to &se them 5 ibraries an% ra strings. e got a farfrom!ompete overvie of feat&res that !an be &se% in reg&ar epression patterns, an% a <&i!- oo- at the range of %ifferent things e !an %o ith them. J&st as reg&ar epressions themseves !an range from simpe to !ompe, so !an their &ses. e !an &se reg&ar epressions for simpe tas-s i-e %etermining hether or not a se<&en!e !ontains a parti!&ar motif, or for !ompi!ate% ones i-e i%entifying messenger ;4 se<&en!es by &sing !ompe patterns. (efore e move on to the eer!ises, its important to re!ognie that for any given pattern, there are probaby m&tipe ays to %es!ribe it &sing a reg&ar epression. ;ear the start of the !hapter, e !ame &p ith the pattern ##+!J",CC to %es!ribe the 4va'' restri!tion enyme re!ognition site, b&t it !o&% aso be ritten as •
##!"2CC%
•
+##!CCJ##"CC,
•
+##!J##",CC
•
#M6!"2CM6
1G
Chapter H: eg&ar epressions
4s ith other sit&ations here there are m&tipe %ifferent ays to rite the same thing, its best to be g&i%e% by hat is !earest to rea%.
1H
Chapter H: eg&ar epressions
!ercises 4ccession names 7eres a ist of ma%e&p gene a!!ession names: -n@B3I, yh%!-2, eih%3@%@, !h%syeIBH, he%e3B, 9h%3e, B%a, %e3H%p rite a program that i print ony the a!!ession names that satisfy the fooing !riteria 5 treat ea!h !riterion separatey: •
!ontain the n&mber
•
!ontain the etter % or e
•
!ontain the etters % an% e in that or%er
•
!ontain the etters % an% e in that or%er ith a singe etter beteen them
•
!ontain both the etters % an% e in any or%er
•
start ith or y
•
start ith or y an% en% ith e
•
!ontain three or more n&mbers in a ro
•
en% ith % fooe% by either a, r or p
'ouble digest 'n the chapter# fie insi%e the eer!ises %onoa%, theres a fie !ae% "na=t!t hi!h !ontains a ma%e&p D;4 se<&en!e. #re%i!t the fragment engths that e i get if e %igest the se<&en!e ith to ma%e&p restri!tion enymes 5 4b!', hose re!ognition site is 4;$W44$, an% 4b!'', hose re!ognition site is RCW$R asteris-s in%i!ate the position of the !&t site=.
1I
Chapter H: eg&ar epressions
&olutions 4ccession names ?bvio&sy, the b&- of the or- here is going to be !oming &p ith the reg&ar epression patterns to see!t ea!h s&bset of the a!!ession names. 7eres the easy bit 5 storing the a!!ession names in a ist an% then pro!essing them in a oop the first ine raps ro&n% be!a&se its too ong to fit on the page=: accs B $*x:nJ'3G*, *yhdc:2*, *eihd3JdJ*, *chdsyeG'7*, *hedle3'*, *xDhd3e*, *'da*, *de37dp* for acc in accs! 4 print if it passes the test
;o e !an ta!-e the %ifferent !riteria one by one. For ea!h eampe, the !o%e bor%ere% by soi% ines= is fooe% imme%iatey by the o&tp&t bor%ere% by %otte% ines=. $he first !riterion is straightforar% 5 a!!essions that !ontain the n&mber . e %ont even have to &se any fan!y reg&ar epression feat&res: for acc in accs! if re.search(r**, acc#! print(*t* C acc#
x:nJ'3G hedle3' xDhd3e 'da
;o for a!!essions that !ontain the etters % or e. e !an &se either aternation or a !hara!ter gro&p. 7eres a so&tion &sing aternation:
1@
Chapter H: eg&ar epressions
for acc in accs! if re.search(r*(dTe#*, acc#! print(*t* C acc#
yhdc:2 eihd3JdJ chdsyeG'7 hedle3' xDhd3e 'da de37dp
$he net one 5 a!!essions that !ontain both the etters % an% e, in that or%er 5 is a bit more tri!-y. e !ant 9&st &se a simpe aternation or a !hara!ter gro&p, be!a&se they mat!h an* of their !onstit&ent parts, an% e nee% bot+ % an% e. ?ne ay to thin- of the pattern is %, fooe% by some other etters an% n&mbers, fooe% by e. e have to be !aref& ith o&r <&antifiers, hoever 5 at first gan!e the pattern d.Fe oo-s goo%, b&t it i fai to mat!h the a!!ession here e foos % %ire!ty. $o ao for the fa!t that % might be imme%iatey fooe% by e, e nee% to &se the asteris-: for acc in accs! if re.search(r*d.Ke*, acc#! print(*t* C acc#
chdsyeG'7 hedle3' xDhd3e de37dp
$he net re<&irement 5 %, fooe% by a singe etter, fooe% by e 5 is a!t&ay easier to rite a pattern for, even tho&gh it so&n%s more !ompi!ate%. e simpy remove the asteris-, an% the perio% i no mat!h any singe !hara!ter:
1G0
Chapter H: eg&ar epressions
for acc in accs! if re.search(r*(d.e#*, acc#! print(*t* C acc#
hedle3'
$he net re<&irement 5 % an% e in any or%er 5 is more %iffi!&t. e !o&% %o it ith an aternation &sing the pattern +d.4eJe.4d,, hi!h transates as " then e0 or e then " . 'n this !ase, ' thin- its !earer to !arry o&t to separate reg&ar epression sear!hes an% !ombine them into a !ompe !on%ition: for acc in accs! if re.search(r*d.Ke*, acc# or re.search(r*e.Kd*, acc#! print(*t* C acc#
hedle3' de37dp
$o fin% a!!essions that start ith either or y, e nee% to !ombine an aternation ith a startofstring an!hor: for acc in accs! if re.search(r*8(xTy#*, acc#! print(*t* C acc#
x:nJ'3G yhdc:2 xDhd3e
1G1
Chapter H: eg&ar epressions
e !an mo%ify this <&ite easiy to a%% the re<&irement that the a!!ession en%s ith e. 4s before, e nee% to &se .W in the mi%%e to mat!h any n&mber of any !hara!ter, res&ting in <&ite a !ompe pattern: for acc in accs! if re.search(r*8(xTy#.Ke6*, acc#! print(*t* C acc#
xDhd3e
$o mat!h three or more n&mbers in a ro, e nee% a more spe!ifi! <&antifier 5 the !&ry bra!-ets 5 an% a !hara!ter gro&p hi!h !ontains a the n&mbers: for acc in accs! if re.search(r*$0123'"7GJU3,100V*, acc#! print(*t* C acc#
x:nJ'3G chdsyeG'7 hedle3'
e !an a!t&ay ma-e this a bit more !on!ise. $he !hara!ter gro&p of a %igits is s&!h a !ommon one that theres a b&itin shorthan% for it: \d. e !an aso ta-e a%vantage of a shorthan% in the !&ry bra!-et <&antifier 5 if e eave off the &pper bo&n%, then it mat!hes ith no &pper imit. $he more !on!ise version: for acc in accs! if re.search(r*dU3,V*, acc#! print(*t* C acc#
1G2
Chapter H: eg&ar epressions
x:nJ'3G chdsyeG'7 hedle3'
$he fina re<&irement is <&ite simpe an% ony re<&ires a !hara!ter gro&p an% an en%ofstring an!hor to sove: for acc in accs! if re.search(r*d$arp6*, acc#! print(*t* C acc#
'da de37dp
'ouble digest $his is a har% probem, an% there are severa ays to approa!h it. +ets simpify it by first fig&ring o&t hat the fragment engths o&% be if e %igeste% the se<&en!e ith 9&st a singe restri!tion enyme1. e open an% rea% the fie a in one go theres no nee% to pro!ess it inebyine as its 9&st a singe se<&en!e=, then e &se re.finditer to fig&re o&t the positions of a the !&t sites. $he patterns themseves are reativey simpe: ; means any base, so the pattern for the 4b!' site is !!"#C2"!!" . $he ambig&ity !o%e Q means ! or # an% the !o%e R means ! or ", so the pattern for 4b!'' is #C!#2!"2"# . 7eres the !o%e to !a!&ate the start positions of the mat!hes for 4b!':
1
For the p&rposes of this eer!ise, e are of !o&rse ignoring a the interesting !hemi!a -ineti!s of restri!tion enymes an% ass&ming that a enymes !&t ith !ompete spe!ifi!ity an% effi!ien!y.
1G3
Chapter H: eg&ar epressions
import re dna B open(*dna.txt*#.read(#.rstrip(*n*# print(*A-cL cuts at!*# for match in re.finditer(r*A$A)%&)AA)*, dna#! print(match.start(##
$he o&tp&t from this oo-s goo%: A-cL cuts at! 11'0 1"2
b&t its not <&ite right 5 its teing &s the positions of the start of ea!h mat!h, b&t the enyme a!t&ay !&ts 3 base pairs &pstream of the start. $o get the position of the !&t site, e nee% to a%% three to the start of ea!h mat!h: import re dna B open(*dna.txt*#.read(#.rstrip(*n*# print(*A-cL cuts at!*# for match in re.finditer(r*A$A)%&)AA)*, dna#! print(match.start(# C 3#
A-cL cuts at! 11'3 1"2G
;o eve got the !&t positions, ho are e going to or- o&t the fragment siesE ?ne ay is to go thro&gh ea!h !&t site in or%er an% meas&re the %istan!e beteen it an% the previo&s one 5 that i give &s the ength of a singe fragment. $o ma-e this or- e have to a%% >imaginary> !&t sites at the very start an% en% of the se<&en!e:
1GB 1 2 3 4 5 6 7
Chapter H: eg&ar epressions
import re dna B open(*dna.txt*#.read(#.rstrip(*n*# all/cuts B $0 for match in re.finditer(r*A$A)%&)AA)*, dna#! all/cuts.append(match.start(# C 3# all/cuts.append(len(dna## print(all/cuts#
+ets ta-e a moment to eamine hats going on in this program. e start by !reating a ne ist variabe !ae% all_cuts to ho% the !&t positions ine 3=. 4t this point, the all_cuts variabe ony has one eement: ero, the position of the start of the se<&en!e. ;et, for ea!h mat!h to the pattern ine B=, e ta-e the start position, a%% three to it to get the !&t position, an% appen% that n&mber to the all_cuts ist ine =. Finay, e appen% the position of the ast !hara!ter in the D;4 string to the all_cuts ist ine G=. hen e print the all_cuts ist, e !an see that it !ontains the position of the start an% en% of the string, an% the interna positions of the !&t sites: $0, 11'3, 1"2G, 2012
;o e !an rite a se!on% oop to go thro&gh the aY!&ts ist an%, for ea!h !&t position, or- o&t the sie of the fragment that i be !reate% by fig&ring o&t the %istan!e to the previo&s !&t site i.e. the previo&s eement in the ist=. $o ma-e this or-, hoever, e !ant 9&st &se a norma oop 5 e have to start at the se!on% eement of the ist be!a&se the first eement has no previo&s eement= an% e have to or- ith the in%e of ea!h eement, rather than the eement itsef. e &se the range f&n!tion to generate the ist of in%ees that e ant to pro!ess 5 e nee% to go from in%e 1 i.e. the se!on% eement of the ist= to the ast in%e hi!h is the ength of the ist=:
1G
1 2 3 4 5
Chapter H: eg&ar epressions
for i in range(1,len(all/cuts##! this/cut/position B all/cuts$i preious/cut/position B all/cuts$iF1 fragment/siEe B this/cut/position F preious/cut/position print(*one fragment siEe is * C str(fragment/siEe##
$he oop variabe i is &se% to store ea!h va&e that is generate% by the range f&n!tion ine 1=. For ea!h va&e of i e get the !&t position at that in%e ine 2= an% the !&t position at the previo&s in%e ine 3= an% then fig&re o&t the %istan!e beteen them ine B=. $he o&tp&t shos ho, for to !&ts, e get three fragments: one fragment siEe is 11'3 one fragment siEe is 'G one fragment siEe is 3G'
;o for the fina part of the so&tion: ho %o e %o the same thing for to %ifferent enymesE e !an a%% in the se!on% enyme pattern ith the appropriate !&t site offset an% appen% the !&t positions to the all_cuts variabe: import re dna B open(*dna.txt*#.read(#.rstrip(*n*# all/cuts B $0 4 add cut positions for A-cL for match in re.finditer(r*A$A)%&)AA)*, dna#! all/cuts.append(match.start(# C 3# 4 add cut positions for A-cLL for match in re.finditer(r*%&$A%$A))%*, dna#! all/cuts.append(match.start(# C '# 4 add the final position all/cuts.append(len(dna## print(all/cuts#
1GG
Chapter H: eg&ar epressions
b&t oo- hat happens hen e print the eements of aY!&ts: $0, 11'3, 1"2G, 'GG, 177, 2012
e get ero, then the to !&t positions for the first enyme in as!en%ing or%er, then the to !&t positions for the se!on% enyme in as!en%ing or%er, then the position of the en% of the se<&en!e. $he metho% for t&rning a ist of !&t positions into fragment sies that e %eveope% above isnt going to or- ith this ist, be!a&se it reies on the ist of positions being in as!en%ing or%er. 'f e try it ith the ist of !&t positions pro%&!e% by the above !o%e, e en% &p ith obvio&sy in!orre!t fragment sies: one one one one one
fragment fragment fragment fragment fragment
siEe siEe siEe siEe siEe
is is is is is
11'3 'G F11'0 10GJ '3'
7appiy, #ythons b&itin sort f&n!tion !an !ome to the res!&e. 4 e nee% to %o is sort the ist of !&t positions before pro!essing it, an% e get the right ansers. 7eres the !ompete, fina !o%e:
1GH
Chapter H: eg&ar epressions
import re dna B open(*dna.txt*#.read(#.rstrip(*n*# print(str(len(dna### all/cuts B $0 4 add cut positions for A-cL for match in re.finditer(r*A$A)%&)AA)*, dna#! all/cuts.append(match.start(# C 3# 4 add cut positions for A-cLL for match in re.finditer(r*%&$A%$A))%*, dna#! all/cuts.append(match.start(# C '# 4 add the final position all/cuts.append(len(dna## sorted/cuts B sorted(all/cuts# print(sorted/cuts#
for i in range(1,len(sorted/cuts##! this/cut/position B sorted/cuts$i preious/cut/position B sorted/cuts$iF1 fragment/siEe B this/cut/position F preious/cut/position print(*one fragment siEe is * C str(fragment/siEe##
1GI
8:
Chapter I: Di!tionaries
'ictionaries
&toring paire" "ata "&ppose e ant to !o&nt the n&mber of 4s in a D;4 se<&en!e. Carrying o&t the !a!&ation is <&ite straightforar% 5 in fa!t its one of the first things e %i% in !hapter 2: dna B *A)&%A)&%A)&%)A&%&)%A* a/count B dna.count(*A*#
7o i o&r !o%e !hange if e ant to generate a !ompete ist of base !o&nts for the se<&en!eE e a%% a ne variabe for ea!h base: dna B *A)&%A)&%A)&%)A&%&)%A* a/count B dna.count(*A*# t/count B dna.count(*)*# g/count B dna.count(*%*# c/count B dna.count(*&*#
an% no o&r !o%e is starting to oo- rather repetitive. 'ts not too ba% for the fo&r in%ivi%&a bases, b&t hat if e ant to generate !o&nts for the 1G %in&!eoti%es: dna B *A)&%A)&%A)&%)A&%&)%A* aa/count B dna.count(*AA*# at/count B dna.count(*A)*# ag/count B dna.count(*A%*# ...etc. etc.
or the GB trin&!eoti%es:
1G@
Chapter I: Di!tionaries
dna B *A)&%A)&%A)&%)A&%&)%A* aaa/count B dna.count(*AAA*# aat/count B dna.count(*AA)*# aag/count B dna.count(*AA%*# ...etc. etc.
For trin&!eoti%es an% onger, the sit&ation is parti!&ary ba%. $he D;4 se<&en!e is 20 bases ong, so it ony !ontains 1I overapping trin&!eoti%es in tota. $his means that e en% &p ith GB %ifferent variabes, at east BG of hi!h i ho% the va&e ero. ?ne possibe ay ro&n% this is to store the va&es in a ist. 'f e &se three neste% oops, e !an generate a possibe trin&!eoti%es, !a!&ate the !o&nt for ea!h one, an% store a the !o&nts in a ist: dna B *AA)%A)&%A)&%)A&%&)%A* all/counts B $ for -ase1 in $A, ), %, &! for -ase2 in $A, ), %, &! for -ase3 in $A, ), %, &! trinucleotide B -ase1 C -ase2 C -ase3 count B dna.count(trinucleotide# print(*count is * C str(count# C * for * C trinucleotide# all/counts.append(count# print(all/counts#
1H0
Chapter I: Di!tionaries
4tho&gh the !o%e is above is <&ite !ompa!t, an% %oesnt re<&ire h&ge n&mbers of variabes, the o&tp&t shos to probems ith this approa!h: count is 0 for AAA count is 1 for AA) count is 0 for AA% count is 0 for AA& count is 0 for A)A count is 0 for A)) count is 1 for A)% count is 2 for A)& W many lines remoed W $0, 1, 0, 0, 0, 0, 1, 2, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 2, 0, 0, 2, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 1, 0, 1, 0, 0, 0, 0
Firsty, the %ata are sti in!re%iby sparse 5 the vast ma9ority of the !o&nts are ero. "e!on%y, the !o&nts themseves are no %is!onne!te% from the trin&!eoti%es. 'f e ant to oo- &p the !o&nt for a singe trin&!eoti%e 5 for eampe, $R4 5 e first have to fig&re o&t that $R4 as the 2 th trin&!eoti%e generate% by o&r oops. ?ny then !an e get the eement at the !orre!t in%e: print(*count for )%A is * C str(all/counts$2'##
1H1
Chapter I: Di!tionaries
e !an try vario&s tri!-s to get ro&n% this probem. hat if e generate% to ists 5 one of !o&nts, an% one of the trin&!eoti%es themsevesE dna B *AA)%A)&%A)&%)A&%&)%A* all/trinucleotides B $ all/counts B $ for -ase1 in $A, ), %, &! for -ase2 in $A, ), %, &! for -ase3 in $A, ), %, &! trinucleotide B -ase1 C -ase2 C -ase3 count B dna.count(trinucleotide# all/trinucleotides.append(trinucleotide# all/counts.append(count# print(all/counts# print(all/trinucleotides#
;o e have to ists of the same ength, ith a onetoone !orrespon%en!e beteen the eements: $0, 1, 0, 0, 0, 0, 1, 2, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 2, 0, 0, 2, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 1, 0, 1, 0, 0, 0, 0 $AAA, AA), AA%, AA&, A)A, A)), A)%, A)&, A%A, A%), A%%, A%&, A&A, A&), A&%, A&&, )AA, )A), )A%, )A&, ))A, ))), ))%, ))&, )%A, )%), )%%, )%&, )&A, )&), )&%, )&&, %AA, %A), %A%, %A&, %)A, %)), %)%, %)&, %%A, %%), %%%, %%&, %&A, %&), %&%, %&&, &AA, &A), &A%, &A&, &)A, &)), &)%, &)&, &%A, &%), &%%, &%&, &&A, &&), &&%, &&&
$his aos &s to oo- &p the !o&nt for a given trin&!eoti%e in a sighty more appeaing ay 5 e !an oo- &p the in%e of the trin&!eoti%e in the all_trinucleotides ist, then get the !o&nt at the same in%e in the all_counts ist:
1H2
Chapter I: Di!tionaries
i B all/trinucleotides.index()%A# c B all/counts$i print(count for )%A is C str(c##
$his is a itte bit ni!er, b&t sti has ma9or %raba!-s. ere sti storing a those eros, an% no e have to ists to -eep tra!- of. e nee% to be in!re%iby !aref& hen manip&ating either of the to ists to ma-e s&re that they stay perfe!ty syn!hronie% 5 if e ma-e any !hange to one ist b&t not the other, then there i no onger be a onetoone !orrespon%en!e beteen eements an% e get the rong anser hen e try to oo- &p a !o&nt. $his approa!h is aso so 1. $o fin% the in%e of a given trin&!eoti%e in the all_trinucleotides ist, #ython has to oo- at ea!h eement one at a time &nti it fin%s the one ere oo-ing for. $his means that as the sie of the ist gros 2, the time ta-en to oo- &p the !o&nt for a given eement i gro aongsi%e it. 'f e ta-e a step ba!- an% thin- abo&t the probem in more genera terms, hat e nee% is a ay of storing pairs of %ata in this !ase, trin&!eoti%es an% their !o&nts= in a ay that aos &s to effi!ienty oo- &p the !o&nt for any given trin&!eoti%e. $his probem of storing paire% %ata is in!re%iby !ommon in programming. e might ant to store: •
protein se<&en!e names an% their se<&en!es
•
D;4 restri!tion enyme names an% their motifs
•
!o%ons an% their asso!iate% amino a!i% resi%&es
•
!oeag&es names an% their emai a%%resses
•
sampe names an% their !oor%inates
•
1 2
or%s an% their %efinitions
4s a r&e, eve avoi%e% ta-ing abo&t performan!e in this boo-, b&t e brea- the r&e in this !ase. For instan!e, imagine !arrying o&t the same eer!ise ith the approimatey one miion &ni<&e 10mers.
1H3
Chapter I: Di!tionaries
4 these are eampes of hat e !a keyBvalue pairs. 'n ea!h !ase e have pairs of keys an% values:
=alue !o&nt protein se<&en!e restri!tion enyme motif amino a!i% resi%&e emai a%%ress !oor%inates %efinition
$he ast eampe in this tabe 5 or%s an% their %efinitions 5 is an interesting one be!a&se e have a too in the physi!a or% for storing this type of %ata 5 a %i!tionary. #ythons too for soving this type of probem is aso !ae% a %i!tionary &s&ay abbreviate% to "ict = an% in this !hapter e see ho to !reate an% &se them.
4reating a "ictionary $he synta for !reating a %i!tionary is simiar to that for !reating a ist, b&t e &se !&ry bra!-ets rather than s<&are ones. )a!h pair of %ata, !onsisting of a -ey an% a va&e, is !ae% an item. hen storing items in a %i!tionary, e separate them ith !ommas. ithin an in%ivi%&a item, e separate the -ey an% the va&e ith a !oon. 7eres a bit of !o%e that !reates a %i!tionary of restri!tion enymes &sing %ata from the previo&s !hapter= ith three items: enEymes B U 9coQL!r%AA))&, AaLL!r%%(AT)#&&, NisL!%&$A)%&%& V
1HB
Chapter I: Di!tionaries
'n this !ase, the -eys an% va&es are both strings 1. "pitting the %i!tionary %efinition over severa ines ma-es it easier to rea%: enEymes B U 9coQL ! r%AA))&, AaLL ! r%%(AT)#&&, NisL ! r%&$A)%&%& V
b&t %oesnt affe!t the !o%e at a. $o retrieve a bit of %ata from the %i!tionary 5 i.e. to oo- &p the motif for a parti!&ar enyme 5 e rite the name of the %i!tionary, fooe% by the -ey in s<&are bra!-ets: print(enEymes$NisL#
$he !o%e oo-s very simiar to &sing a ist, b&t instea% of giving the in%e of the eement e ant, ere giving the key for the value that e ant to retrieve. Di!tionaries are a very &sef& ay to store %ata, b&t they !ome ith some restri!tions. $he ony types of %ata e are aoe% to &se as -eys are strings an% n&mbers2, so e !ant, for eampe, !reate a %i!tionary here the -eys are fie ob9e!ts. Sa&es !an be hatever type of %ata e i-e. 4so, -eys must be &ni<&e 5 e !ant store m&tipe va&es for the same -ey. 'n reaife programs, its reativey rare that e ant to !reate a %i!tionary a in one go i-e in the eampe above. More often, e ant to !reate an empty %i!tionary, then a%% -ey/va&e pairs to it 9&st as e often !reate an empty ist an% then a%% eements to it=. $o !reate an empty %i!tionary e simpy rite a pair of !&ry bra!-ets on their on, an% to a%% eements, e &se the s<&arebra!-ets notation on the efthan% si%e of 1 2
$he va&es are a!t&ay ra strings, b&t thats not important. ;ot stri!ty tr&e e !an &se any imm&tabe type, b&t that is beyon% the s!ope of this boo-.
1H
Chapter I: Di!tionaries
an assignment. 7eres a bit of !o%e that stores the restri!tion enyme %ata one item at a time: enEymes B UV enEymes$9coQL B r%AA))& enEymes$AaLL B r%%(AT)#&& enEymes$NisL B r%&$A)%&%&
e !an %eete a -ey from a %i!tionary &sing the pop metho%. pop a!t&ay ret&rns the va&e an% %eetes the -ey at the same time: enEymes B U 9coQL ! r%AA))&, AaLL ! r%%(AT)#&&, NisL ! r%&$A)%&%& V 4 remoe the 9coQL enEyme from the dict enEymes.pop(9coQL#
+ets ta-e another oo- at the trin&!eoti%e !o&nt eampe from the start of the !hapter. 7eres ho e store the trin&!eoti%es an% their !o&nts in a %i!tionary: dna B *AA)%A)&%A)&%)A&%&)%A* counts B UV for -ase1 in $A, ), %, &! for -ase2 in $A, ), %, &! for -ase3 in $A, ), %, &! trinucleotide B -ase1 C -ase2 C -ase3 count B dna.count(trinucleotide# counts$trinucleotide B count print(counts#
e !an see from the o&tp&t that the trin&!eoti%es an% their !o&nts are store% together in one variabe:
1HG
Chapter I: Di!tionaries
UA&&! 0, A)%! 1, AA%! 0, AAA! 0, A)&! 2, AA&! 0, A)A! 0, A%%! 0, &&)! 0, &)&! 0, A%&! 0, A&A! 0, A%A! 0, &A)! 0, AA)! 1, A))! 0, &)%! 1, &)A! 0, A&)! 0, &A&! 0, A&%! 1, &AA! 0, A%)! 0, &A%! 0, &&%! 0, &&&! 0, &))! 0, )A)! 0, %%)! 0, )%)! 0, &%A! 1, &&A! 0, )&)! 0, %A)! 2, &%%! 0, )))! 0, )%&! 0, %%%! 0, )A%! 0, %%A! 0, )AA! 0, %%&! 0, )A&! 1, ))&! 0, )&%! 2, ))A! 0, ))%! 0, )&&! 0, %AA! 0, )%%! 0, %&A! 0, %)A! 1, %&&! 0, %)&! 0, %&%! 0, %)%! 0, %A%! 0, %))! 0, %&)! 1, )%A! 2, %A&! 0, &%)! 1, )&A! 0, &%&! 1V
e sti have a ot of repetitive !o&nts of ero, b&t oo-ing &p the !o&nt for a parti!&ar trin&!eoti%e is no very straightforar%: print(counts$)%A#
e no onger have to orry abo&t either >memoriing> the or%er of the !o&nts or maintaining to separate ists. +ets no see if e !an fin% a ay of avoi%ing storing a those ero !o&nts. e !an a%% an if statement that ens&res that e ony store a !o&nt if its greater than ero: dna B *AA)%A)&%A)&%)A&%&)%A* counts B UV for -ase1 in $A, ), %, &! for -ase2 in $A, ), %, &! for -ase3 in $A, ), %, &! trinucleotide B -ase1 C -ase2 C -ase3 count B dna.count(trinucleotide# if count + 0! counts$trinucleotide B count print(counts#
1HH
Chapter I: Di!tionaries
hen e oo- at the o&tp&t from the above !o%e, e !an see that the amo&nt of %ata ere storing is m&!h smaer 5 9&st the !o&nts for the trin&!eoti%es that are greater than ero: UA)%! 1, A&%! 1, A)&! 2, %)A! 1, &)%! 1, &%&! 1, %A)! 2, &%A! 1, AA)! 1, )%A! 2, %&)! 1, )A&! 1, )&%! 2, &%)! 1V
;o e have a ne probem to %ea ith. +oo-ing &p the !o&nt for a given trin&!eoti%e or-s fine hen the !o&nt is positive: print(counts$)%A#
(&t hen the !o&nt is ero, the trin&!eoti%e %oesnt appear as a -ey in the %i!tionary: print(counts$AAA#
so e i get a Key)rror hen e try to oo- it &p: Iey9rror! AAA
$here are to possibe ays to fi this. e !an !he!- for the eisten!e of a -ey in a %i!tionary 9&st i-e e !an !he!- for the eisten!e of an eement in a ist=, an% ony try to retrieve it on!e e -no it eists: if AAA in counts! print(counts(AAA##
1HI
Chapter I: Di!tionaries
4ternativey, e !an &se the %i!tionarys get metho%. get &s&ay or-s 9&st i-e &sing s<&are bra!-ets: the fooing to ines %o ea!ty the same thing: print(counts$)%A# print(counts.get()%A##
$he thing that ma-es get reay &sef&, hoever, is that it !an ta-e an optiona se!on% arg&ment, hi!h is the %efa&t va&e to be ret&rne% if the -ey isnt present in the %i!tionary. 'n this !ase, e -no that if a given trin&!eoti%e %oesnt appear in the %i!tionary then its !o&nt is ero, so e !an give ero as the %efa&t va&e an% &se get to print o&t the !o&nt for any trin&!eoti%e: print(*count print(*count print(*count print(*count
for for for for
)%A AAA %)A )))
is is is is
* * * *
C C C C
str(counts.get()%A, str(counts.get(AAA, str(counts.get(%)A, str(counts.get())),
0### 0### 0### 0###
4s e !an see from the o&tp&t, e no %ont have to orry abo&t hether or not ea!h trin&!eoti%e appears in the %i!tionary 5 get ta-es !are of everything an% ret&rns ero hen appropriate: count count count count
for for for for
)%A AAA %)A )))
is is is is
2 0 1 0
1H@
Chapter I: Di!tionaries
terating over a "ictionary hat if, instea% of oo-ing &p a singe item from a %i!tionary, e ant to %o something for a itemsE For eampe, imagine that e ante% to ta-e o&r !o&nts %i!tionary variabe from the !o%e above an% print o&t a trin&!eoti%es here the !o&nt as 2. ?ne ay to %o it o&% be to &se o&r three neste% oops again to generate a possibe trin&!eoti%es, then oo- &p the !o&nt for ea!h one an% %e!i%e hether or not to print it: for -ase1 in $A, ), %, &! for -ase2 in $A, ), %, &! for -ase3 in $A, ), %, &! trinucleotide B -ase1 C -ase2 C -ase3 if counts.get(trinucleotide, 0# BB 2! print(trinucleotide#
4s e !an see from the o&tp&t, this or-s perfe!ty e: A)& )%A )&% %A)
(&t it seems ineffi!ient to go thro&gh the hoe pro!ess of generating a possibe trin&!eoti%es again, hen the information e ant 5 the ist of trin&!eoti%es 5 is area%y in the %i!tionary. 4 better approa!h o&% be to rea% the ist of -eys %ire!ty from the %i!tionary, hi!h is hat the keys metho% %oes.
Iterating over 0e*s hen &se% on a %i!tionary, the keys metho% ret&rns a ist of a the -eys in the %i!tionary:
1I0
Chapter I: Di!tionaries
print(counts.:eys(##
+oo-ing at the o&tp&t 1 !onfirms that this is the ist of trin&!eoti%es e ant to !onsi%er remember that ere oo-ing for trin&!eoti%es ith a !o&nt of to, so e %ont nee% to !onsi%er ones that arent in the %i!tionary as e area%y -no that they have a !o&nt of ero=: $A)%, A&%, A)&, %)A, &)%, &%&, %A), &%A, AA), )%A, %&), )A&, )&%, &%)
*sing keys, o&r !o%e for printing o&t a the trin&!eoti%es that appear ti!e in the D;4 se<&en!e be!omes a ot more !on!ise: for trinucleotide in counts.:eys(#! if counts.get(trinucleotide# BB 2! print(trinucleotide#
$his version prints ea!ty the same set of trin&!eoti%es as the more verbose metho%: A)& %A) )%A )&%
(efore e move on, ta-e a moment to !ompare the o&tp&t imme%iatey above this paragraph ith the o&tp&t from the threeoop version from earier in this se!tion. o& noti!e that hie the s et of trinucleotides is the same, the order in +ic+ t+e* appear is %ifferent. $his i&strates an important point abo&t %i!tionaries 5 they are inherently unor"ere" . $hat means that hen e &se the keys metho% to 1
'f yo&re &sing #ython 3 yo& might see sighty %ifferent o&tp&t here, b&t a the !o%e eampes i or9&st the same
1I1
Chapter I: Di!tionaries
iterate over a %i!tionary, e !ant rey on pro!essing the items in the same or%er that e a%%e% them. $his is in !ontrast to ists, hi!h aays maintain the same or%er hen ooping. 'f e ant to !ontro the or%er in hi!h -eys are printe% e !an &se the sorted metho% to sort the ist before pro!essing it: for trinucleotide in sorted(counts.:eys(##! if counts.get(trinucleotide# BB 2! print(trinucleotide#
Iterating over items 'n the eampe !o%e above, the first thing e nee% to %o insi%e the oop is to oo&p the va&e for the !&rrent -ey. $his is a very !ommon pattern hen iterating over %i!tionaries 5 so !ommon, in fa!t, that #ython has a spe!ia shorthan% for it. 'nstea% of %oing this: for :ey in my/dict.:eys(#! alue B my/dict.get(:ey# 4 do something ith :ey and alue
e !an &se the items metho% to iterate over pairs of %ata, rather than 9&st -eys: for :ey, alue in my/dict.items(#! 4 do something ith :ey and alue
$he items metho% %oes something sighty %ifferent from a the other metho%s eve seen so far in this boo- rather than ret&rning a single value, or a list of values, it ret&rns a list of pairs of values . $hats hy e have to give to variabe names at the start of the oop. 7eres ho e !an &se the items metho% to pro!ess o&r %i!tionary of trin&!eoti%e !o&nts 9&st i-e before:
1I2
Chapter I: Di!tionaries
for trinucleotide, count in counts.items(#! if count BB 2! print(trinucleotide#
$his metho% is generay preferre% for iterating over items in a %i!tionary, as it ma-es the intention of the !o%e very !ear.
(ecap e starte% this !hapter by eamining the probem of storing paire% %ata in #ython. 4fter oo-ing at a !o&pe of &nsatisfa!tory ays to %o it &sing toos that eve area%y earne% abo&t, e intro%&!e% a ne type of %ata str&!t&re 5 the %i!tionary 5 hi!h offers a m&!h ni!er so&tion to the probem of storing paire% %ata. +ater in the !hapter, e sa that the rea benefit of &sing %i!tionaries is the effi!ient oo-&p they provi%e. e sa ho to !reate %i!tionaries an% manip&ate the items in them, an% severa %ifferent ays to oo- &p va&es for -non -eys. e aso sa ho to iterate over a the items in %i!tionary. 'n the pro!ess, e &n!overe% a fe restri!tions on hat %i!tionaries are !apabe of 5 ere ony aoe% to &se a !o&pe of %ifferent %ata types for -eys, they m&st be &ni<&e, an% e !ant rey on their or%er. J&st as a physi!a %i!tionary aos &s to rapi%y oo- &p the %efinition for a or% b&t not the other ay ro&n%, #ython %i!tionaries ao &s to rapi%y oo- &p the va&e asso!iate% ith a -ey, b&t not the reverse.
1I3
Chapter I: Di!tionaries
!ercises '.4 translation rite a program that i transate a D;4 se<&en!e into protein. o&r program sho&% &se the stan%ar% geneti! !o%e hi!h !an be fo&n% at this *+ 1.
1
http://.n!bi.nm.nih.gov/$aonomy/taonomyhome.htm/in%e.!giE!hapterZtgen!o%es["R1
1IB
Chapter I: Di!tionaries
&olutions '.4 translation $he %es!ription of this eer!ise is very short, b&t it hi%es <&ite a bit of !ompeity8 $o transate a D;4 se<&en!e e nee% to !arry o&t a n&mber of %ifferent steps. First, e have to spit &p the se<&en!e into !o%ons. $hen, e nee% to go thro&gh ea!h !o%on an% transate it into the !orrespon%ing amino a!i% resi%&e. Finay, e nee% to !reate a protein se<&en!e by a%%ing a the amino a!i% resi%&es together. e start off by fig&ring o&t ho to spit a D;4 se<&en!e into !o%ons. (e!a&se this eer!ise is <&ite tri!-y, e pi!- a very short test D;4 se<&en!e to or- on 5 9&st three !o%ons: dna B *A)%))&%%)*
7o are e going to spit &p the D;4 se<&en!e into gro&ps of three basesE 'ts tempting to try to &se the split metho%, b&t remember that the split metho% ony or-s if the things yo& ant to spit are separate% by a %eimiter. 'n o&r !ase, theres nothing separating the !o%ons, so split i not hep &s. "omething that might be abe to hep &s is s&bstring notation. e -no that this aos &s to etra!t part of a string, so e !an %o something i-e this: dna B *A)%))&%%)* codon1 B dna$0!3 codon2 B dna$3!" codon3 B dna$"!J print(codon1, codon2, codon3#
4s e !an see from the o&tp&t, this or-s:
1I
Chapter I: Di!tionaries
(A)%, ))&, %%)#
b&t its not a great so&tion, as e have to fi in the n&mbers man&ay. "in!e the n&mbers foo a very pre%i!tabe pattern, it sho&% be possibe to generate them a&tomati!ay. $he start position for ea!h s&bstring is initiay ero, then goes &p by three for ea!h s&!!essive !o%on. $he stop position is 9&st the start position p&s three. e!a that the 9ob of the range f&n!tion is to generate se<&en!es of n&mbers. 'n or%er to generate the se<&en!e of s&bstring start positions, e nee% to &se the threearg&ment version of range, here the first arg&ment is the n&mber to start at, the se!on% arg&ment is the n&mber to finish at, an% the thir% arg&ment is the step sie. For o&r D;4 se<&en!e above, the n&mber to start at is ero, an% the step sie is three. $he n&mber to finish at it not si b&t seven, be!a&se ranges are e!&sive at the finish. $his bit of !o%e shos ho e !an &se the range f&n!tion to generate the ist of start positions: for start in range(0,7,3#! print(start#
0 3 "
$o fin% the stop position for a given start position e 9&st a%% three, so e !an easiy spit o&r D;4 into !o%ons &sing a oop: dna B *A)%))&%%)* for start in range(0,7,3#! codon B dna$start!startC3 print(*one codon is* C codon#
1IG
Chapter I: Di!tionaries
one codon is A)% one codon is ))& one codon is %%)
$his or-s fine for o&r test D;4 se<&en!e, b&t if e give it a shorter se<&en!e e i get in!ompete an% empty !o%ons: dna B *A)%))* for start in range(0,7,3#! codon B dna$start!startC3 print(codon#
one codon is A)% one codon is )) one codon is
an% if e give it a onger se<&en!e, e i miss o&t the fo&rth an% s&bse<&ent !o%ons: dna B *A)%))&%%)%AA%&%%%&)A%A)* for start in range(0,7,3#! codon B dna$start!startC3 print(*one codon is * C codon#
one codon is A)% one codon is ))& one codon is %%)
Ceary e nee% to mo%ify the se!on% arg&ment to range 5 the position to finish the se<&en!e of n&mbers 5 in or%er to ta-e into a!!o&nt the ength of the D;4 se<&en!e. 4t this point, e have to !onfront the probem of hat to %o if ere given a D;4 se<&en!e hose ength is not an ea!t m&tipe of three. Ceary, e !annot transate an in!ompete !o%on, so e ant the start position of the fina
1IH
Chapter I: Di!tionaries
!o%on to e<&a to the ength of the D;4 se<&en!e min&s to. $his g&arantees that there i aays be to more !hara!ters fooing the position of the fina !o%on start 5 i.e. eno&gh for a !ompete !o%on. 7eres the mo%ifie% !o%e: dna B *A)%))&%%)* 4 calculate the start position for the final codon last/codon/start B len(dna# X 2 4 process the dna seuence in three -ase chun:s for start in range(0,last/codon/start,3#! codon B dna$start!startC3 print(*one codon is * C codon#
;o that e -no ho to spit a D;4 se<&en!e &p into !o%ons, ets t&rn o&r attention to the probem of transating those !o%ons. 'f e p& &p the *+ from the eer!ise %es!ription in a eb broser, e !an see the stan%ar% !o%on transation tabe in vario&s formats. "toring this transation tabe seems i-e a perfe!t 9ob for a %i!tionary: e have !o%ons -eys= an% amino a!i% resi%&es va&es= an% e ant to be abe to oo- &p the amino a!i% for a given !o%on.
1II
Chapter I: Di!tionaries
7eres a bit of !o%e 5 its a!t&ay a singe statement, sprea% o&t over m&tipe ines 5 hi!h !reates a %i!tionary to ho% the transation tabe: gencode @ M )!"!)0);)% )!"C)0);)% )!C!)0)")% )!CC)0)")% )!!C)0)N)% )!!")0)N)% )!#C)0)S)% )!#")0)S)% )C"!)0)T)% )C"C)0)T)% )CC!)0)P)% )CCC)0)P)% )C!C)0):)% )C!")0):)% )C#!)0)Q)% )C#C)0)Q)% )#"!)0)V)% )#"C)0)V)% )#C!)0)!)% )#CC)0)!)% )#!C)0)()% )#!")0)()% )##!)0)#)% )##C)0)#)% )"C!)0)S)% )"CC)0)S)% )""C)0)7)% )""")0)7)% )"!C)0)')% )"!")0)')% )"#C)0)C)% )"#")0)C)%
)!"")0);)% )!C#)0)")% )!!!)0))% )!#!)0)Q)% )C"#)0)T)% )CC#)0)P)% )C!!)0)U)% )C##)0)Q)% )#"#)0)V)% )#C#)0)!)% )#!!)0)E)% )###)0)#)% )"C#)0)S)% )""!)0)T)% )"!!)0)_)% )"#!)0)_)%
)!"#)0)&)% )!C")0)")% )!!#)0))% )!##)0)Q)% )C"")0)T)% )CC")0)P)% )C!#)0)U)% )C#")0)Q)% )#"")0)V)% )#C")0)!)% )#!#)0)E)% )##")0)#)% )"C")0)S)% )""#)0)T)% )"!#)0)_)% )"##)0)R)
e !an oo- &p the amino a!i% for a given !o%on !o%on &sing either of the to metho%s that e earne% abo&t: print(gencode$&A)# print(gencode.get(%)&##
1I@
Chapter I: Di!tionaries
'f e oo- &p the amino a!i% for ea!h !o%on insi%e the oop of o&r origina !o%e, e e !an print both the !o%on an% the amino a!i% transation 1: dna B *A)%))&%%)* last/codon/start B len(dna# F 2 for start in range(0,last/codon/start,3#! codon B dna$start!startC3 aa B gencode.get(codon# print(*one codon is * C codon# print(*the amino acid is * C aa#
one the one the one the
codon amino codon amino codon amino
is A)% acid is is ))& acid is is %%) acid is %
$his is starting to oo- promising. $he fina step is to a!t&ay %o something ith the amino a!i% resi%&es rather than 9&st printing them. 4 ni!e ni!e i%ea is to ta-e o&r !&e from the ay that a ribosome behaves an% a%% ea!h ne amino a!i% resi%&e onto the en% of a protein to !reate a gra%&aygroing string:
1
From no no on, on, e ont ont in!&%e in!&%e the the statemen statementt hi!h !reate !reatess the %i!tion %i!tionary ary in o&r o&r !o%e samp sampes es as it ta-es ta-es &p too m&!h room, so if yo& ant to try r&nning these yo&rsef yo& nee% to a%% it ba!- at the top.
1@0
1 2 3 4 5 6 7 8
Chapter I: Di!tionaries
dna B *A)%))&%%)* last/codon/start B len(dna# F 2 protein B ** for start in range(0,last/codon/start,3#! codon B dna$start!startC3 aa B gencode.get(codon# protein B protein C aa print(*protein seuence is * C protein#
'n the above !o%e, e !reate a ne variabe to ho% the protein se<&en!e imme%iatey before e start the oop ine 3=, then a%% a singe !hara!ter onto the en% of that variabe ea!h time ro&n% the oop ine H=. (y the time e eit the oop, e have b&it b&it &p the !ompete protein protein se<&en!e an% e e !an print it o&t ine ine I=: protein seuence is %
$his oo-s i-e a very &sef& bit of !o%e, so ets t&rn it into a f&n!tion. ?&r f&n!tion i ta-e one arg&ment arg&ment 5 the D;4 se<&en!e se<&en!e as a string 5 an% i ret&rn a string !ontaining the protein se<&en!e 1: def translate/dna(dna#! last/codon/start B len(dna# F 2 protein B ** for start in range(0,last/codon/start,3#! codon B dna$start!startC3 aa B gencode.get(codon# protein B protein C aa return protein
e !an no test o&r f&n!tion by printing o&t the protein transation for a fe more test se<&en!es: 1
o& o& not noti!e i!e that that this this f&n!t f&n!tion ion reies reies on the the gencode variabe hi!h is %efine% o&tsi%e the f&n!tion 5 something that ' to% yo& not to %o in !hapter . $his is an e!eption to the r&e: %efining the gen!o%e variabe insi%e the the f&n!tion means means that it o&% o&% have to be !reate% ane ane ea!h time e ante% to transate a D;4 se<&en!e.
1@1
Chapter I: Di!tionaries
print(translate/dna(*A)%))&%%)*## print(translate/dna(*A)&%A)&%A)&%))%&))A)&%A)&A%*## print(translate/dna(*actgatcgtagctagctgacgtatcgtat*## print(translate/dna(*A&%A)&%A)&%)
$he o&tp&t from this !o%e shos that e r&n into a probem ith the thir% se<&en!e: % L@QS??L@Y )race-ac: (most recent call last#! ile *dna/translation.py*, line 30, in ;module+ print(translate/dna(*actgatcgtagctagctgacgtatcgtat*## ile *dna/translation.py*, line 2, in translate/dna protein B protein C aa )ype9rror! cannot concatenate str and
$he probem o!!&rs hen e try to oo- &p the amino a!i% for the first !o%on of the thir% se<&en!e 5 >a!t>. (e!a&se (e!a&se the thir% t hir% se<&en!e is in oer !ase b&t the transation tabe %i!tionary is in &pper !ase, the -ey isnt fo&n%, the get metho% ret&rns None, an% e get an error. Fiing it is straightforar% 5 e 9&st nee% to !onvert the !o%on !o%on to &pper !ase before oo-ing &p the amino a!i%: def translate/dna(dna#! last/codon/start B len(dna# F 2 protein B ** for start in range(0,last/codon/start,3#! codon B dna$start!startC3 aa B gencode.get(codon.upper(## protein B protein C aa return protein
;o the o&tp&t shos that the first three se<&en!es are fine, b&t that o&r f&n!tion has a probem transating the fo&rth se<&en!e:
1@2
Chapter I: Di!tionaries
% L@QS??L@Y )@QS??)RQ )race-ac: (most recent call last#! ile *dna/translation.py*, line 31, in ;module+ print(translate/dna(*A&%A)&%A)&%)
Ran!ing at the inp&t se<&en!es, its not !ear hat the probem is. +ets try printing the !o%ons as theyre transate% in or%er to i%entify the one thats !a&sing the error: def translate/dna(dna#! last/codon/start B len(dna# F 2 protein B ** for start in range(0,last/codon/start,3#! codon B dna$start!startC3 print(*a-out to translate codon! * C codon# aa B gencode.get(codon.upper(## protein B protein C aa return protein print(translate/dna(*A&%A)&%A)&%)
$he o&tp&t shos here the probem ies:
1@3
Chapter I: Di!tionaries
a-out to translate codon! A&% a-out to translate codon! A)& a-out to translate codon! %A) a-out to translate codon! &%) a-out to translate codon!
$here is an &n-non base in the mi%%e of the D;4 se<&en!e, hi!h !a&ses o&r f&n!tion to try to oo- &p the amino a!i% for the !o%on N!C, hi!h !a&ses an error be!a&se that !o%on isnt in the %i!tionary. 7o sho&% e fi thisE e !o&% a%% an if statement to the f&n!tion hi!h ony transates the D;4 se<&en!e if it %oesnt !ontain any &nambig&o&s bases, b&t that seems a itte too !onservative 5 there are penty of sit&ations in hi!h e might ant to generate a protein se<&en!e for a D;4 se<&en!e that has &n-non bases. e !o&% a%% an if statement insi%e the oop hi!h ony transates a given !o%on if it %oesnt !ontain any &nambig&o&s bases, b&t that o&% ea% to protein transations of an in!orre!t ength 5 e -no that the !o%on ;4C i transate to an amino a!i%, e 9&st %ont -no hi!h one it i be. $he most sensibe so&tion seems to be to transate any !o%on ith an &n-non base into the symbo for an &n-non amino a!i% resi%&e, hi!h is W. $he optiona se!on% arg&ment to the get f&n!tion ma-es it very easy to %o 9&st that:
1@B
Chapter I: Di!tionaries
def translate/dna(dna#! last/codon/start B len(dna# F 2 protein B ** for start in range(0,last/codon/start,3#! codon B dna$start!startC3 aa B gencode.get(codon.upper(#, Z# protein B protein C aa return protein
an% no e !an transate a fo&r of o&r test se<&en!es !orre!ty: print(translate/dna(*A)%))&%%)*## print(translate/dna(*A)&%A)&%A)&%))%&))A)&%A)&A%*## print(translate/dna(*actgatcgtagcttgcttacgtatcgtat*## print(translate/dna(*A&%A)&%A)&%)
% L@QS??L@Y )@QS??)RQ )L@QZQSRS
4t this point, its a goo% i%ea to t&rn these test se<&en!es into assert statements 5 that ay, e !an easiy retest the f&n!tion if e ma-e some !hanges to it in the f&t&re: assert(translate/dna(*A)%))&%%)*## BB *%* assert(translate/dna(*A)&%A)&%A)&%))%&))A)&%A)&A%*## BB *L@QS??L@Y* assert(translate/dna(*actgatcgtagcttgcttacgtatcgtat*## BB *)@QS??)RQ* assert(translate/dna(*A&%A)&%A)&%)
1@
Chapter @: Fies, programs, an% &ser inp&t
9:
(iles) programs) and user input
/ile contents an" manipulation ea%ing from an% riting to fies as one of the first things e oo-e% at in this boo-, ba!- in !hapter 3. For some programs, hoever, ere not 9&st !on!erne% ith the !ontents of fies, b&t ith fies an% fo%ers themseves. $his is espe!iay i-ey to be the !ase for programs that have to operate as part of a or- fo invoving other toos an% softare. For eampe, e may nee% to !opy, move, rename an% %eete fies, or e may nee% to pro!ess a fies in a !ertain fo%er. 4tho&gh it seems i-e a simpe tas- after a, the fie manager toos that !ome ith yo&r operating system !an !arry most of them o&t=, fie manip&ation in a ang&age i-e #ython is a!t&ay <&ite tri!-y. $hats be!a&se the !o%e that e rite has to f&n!tion i%enti!ay on %ifferent operating systems 5 in!&%ing in%os, +in& an% Ma! ma!hines 5 hi!h may han%e fies <&ite %ifferenty. 4 %is!&ssion of the %ifferen!es beteen operating systems is ay beyon% the s!ope of this boo-, b&t to give one eampe, *;'base% systems i-e +in& an% ?" have the !on!ept of fie permissions hi!h is a!-ing in in%os. $han-f&y, #ython in!&%es a !o&pe of mo%&es 1 that ta-e !are of these %ifferen!es for &s an% provi%e &s ith a set of &sef& f&n!tions for manip&ating fies. $he mo%&es names are os short for ?perating "ystem= an% shutil short for "7e *$'+ities=. 'n the net se!tion e see ho they !an be &se% to !arry o&t vario&s !ommon b&t important= tas-s.
4 note on t+e code examples "in!e the !o%e eampes in this !hapter &navoi%aby invove intera!tion ith the operating system, some of the %etais i be operatingsystem spe!ifi!. 'n 1
$a-e a oo- ba!- at !hapter H for a remin%er of ho mo%&es or-.
1@G
Chapter @: Fies, programs, an% &ser inp&t
parti!&ar, many of the fie manip&ation f&n!tions ta-e paths as arg&ments, hi!h %iffer !onsi%eraby beteen operating systems. 4 path is the short bit of tet that tes yo& the o!ation of a fie in the fie system. ?n +in& an% ?" ma!hines, the path to a fie or fo%er typi!ay oo-s i-e this: pathtomyfile.txt
hereas on in%os ma!hines, they oo- i-e this: c!pathtomyfile.txt
Moreover, the s&!!ess of the !o%e eampes for many f&n!tions reies on the fies an% fo%ers a!t&ay being present on the !omp&ter on hi!h the eampes are r&n. $he !o%e eampes in this !hapter i &se +in&stye paths, an% i refer to fo%ers an% fies on my !omp&ter, so if yo& ant to try r&nning them, yo& probaby nee% to !hange the paths to refer to fies on yo&r on !omp&ter.
8asic file manipulation $o rename an eisting fie, e simpy import the os mo%&e, then &se the os.rename f&n!tion. $he os.rename f&n!tion ta-es to arg&ments, both strings. $he first is the !&rrent name of the fie, the se!on% is the ne name: import os os.rename(*old.txt*, *ne.txt*#
$he above !o%e ass&mes that the fie ol"=t!t is in the fo%er here e are r&nning o&r #ython program. 'f its esehere in the fiesystem, then e have to give the !ompete path: os.rename(*homemartin-iologyold.txt*, *homemartin-iolgyne.txt*#
1@H
Chapter @: Fies, programs, an% &ser inp&t
'f e spe!ify a %ifferent fo%er, b&t the same fie name, in the se!on% arg&ment, then the f&n!tion i move the fie from one fo%er to another: os.rename(*homemartin-iologyold.txt*, *homemartinpythonold.txt*#
?f !o&rse, e !an move an% rename a fie in one step if yo& i-e: os.rename(*homemartin-iologyold.txt*, *homemartinpythonne.txt*#
os.rename or-s on fo%ers as e as fies: os.rename(*homemartinold/folder*, *homemartinne/folder*#
'f e try to move a fie to a fo%er that %oesnt eist e get an error. e nee% to !reate the ne fo%er first ith the os.mkdir f&n!tion: os.m:dir(*homemartinpython*#
'f e nee% to !reate a b&n!h of %ire!tories a in one go, e !an &se the os.mkdirs f&n!tion note the s on the en% of the name=: os.m:dir(*alongpathithlotsoffolders*#
$o !opy a fie or fo%er e &se the shutil mo%&e. e !an !opy a singe fie ith shutil.copy: shutil.copy(*homemartinoriginal.txt*, *homemartincopy.txt*#
or a fo%er ith shutil.copytree :
1@I
Chapter @: Fies, programs, an% &ser inp&t
shutil.copytree(*homemartinoriginal/folder*, *homemartincopy/folder*#
$o test hether a fie or fo%er eists, &se os.path.exists : if os.path.exists(*homemartinemail.txt*#! print(*Rou hae mail5*#
eleting files an" fol"ers $here are %ifferent f&n!tions for %eeting fies, empty fo%ers, an% nonempty fo%ers. $o %eete a singe fie, &se os.remoe: os.remoe(*homemartinunanted/file.txt*#
$o %eete an empty fo%er, &se os.rmdir : os.rmdir(*homemartinemtpy*#
$o %eete a fo%er an% a the fies in it, &se shutil.rmtree shutil.rmtree(*homemartinfull*#
6isting fol"er contents $he os.listdir f&n!tion ret&rns a ist of fies an% fo%ers. 't ta-es a singe arg&ment hi!h is a string !ontaining the path of the fo%er hose !ontents yo& ant to sear!h. $o get a ist of the !ontents of the !&rrent or-ing %ire!tory, &se the string >.> for the path:
1@@
Chapter @: Fies, programs, an% &ser inp&t
for file/name in os.listdir(*.*#! print(*one file name is * C file/name#
$o ist the !ontents of a %ifferent fo%er, e 9&st give the path as an arg&ment: for file/name in os.listdir(*homemartin*#! print(*one file name is * C file/name#
(unning e!ternal programs 4nother feat&re of #ython that invoves intera!tion ith the operating system is the abiity to r&n eterna programs. J&st i-e fie an% fo%er manip&ation, the abiity to r&n other programs is very &sef& hen &sing #ython as part of a orfo. 't aos &s to &se eisting toos that o&% be very time!ons&ming to re!reate in #ython, or that o&% r&n very soy. &nning eterna programs from ithin yo&r #ython !o%e !an be a tri!-y b&siness, an% this feat&re o&%nt normay be !overe% in an intro%&!tory programming !o&rse. 7oever, its so &sef& for bioogy an% s!ien!e in genera= that ere going to !over it here, abeit in a simpifie% form. 4s ith the above se!tion on fie operations, the ea!t %etais of ho eterna programs are r&n i vary ith yo&r operating system an% the ay yo&r !omp&ter is set &p. ?n *;'base% systems, the program that yo& ant to r&n might area%y be in yo&r path, in hi!h !ase yo& !an simpy &se the name of the ee!&tabe as the string to be ee!&te%. For the eampe !o%e beo, ' give the f& path to ee!&tabes on my !omp&ter, hi!h oo- something i-e this: homemartinsoftaremyprogram
'f yo&re on in%os, yo&r paths i probaby oo- i-e this:
200
Chapter @: Fies, programs, an% &ser inp&t
c!indosProgram filesmyprogrammyprogram.exe
4n% on ?", they i oo- i-e this: Applicationsmyprogram
4s before, if yo& ant to try r&nning any of these eampes, ma-e s&re that yo& !hange the paths to point to rea ee!&tabes on yo&r !omp&ter.
(unning a program $he f&n!tions for r&nning eterna program resi%e in the su$process mo%&e. $he reasoning behin% the name is sighty !onvo&te%: hen ta-ing abo&t operating systems, a r&nning program is !ae% a process, an% a program that is starte% by another program is !ae% a subprocess. $o r&n an eterna program, &se the su$process.call f&n!tion. $his f&n!tion ta-es a singe string arg&ment !ontaining the path to the ee!&tabe yo& ant to r&n: import su-process su-process.call(*-indate*#
4ny o&tp&t that is pro%&!e% by the eterna program is printe% straight to the s!reen 5 in this !ase, the o&tp&t from the +in& date program: ri Hul 2" 1!1!2" NS) 2013
'f e ant to s&ppy !omman%ine options to the eterna program then e 9&st in!&%e them in the string, an% set the optiona shell arg&ment to "rue. 7ere e !a the +in& date program ith the options hi!h !a&se it to 9&st print the month:
201
Chapter @: Fies, programs, an% &ser inp&t
su-process.call(*-indate C[N*, shellB)rue#
Huly
&aving program output ?ften, e ant to r&n some eterna program an% then store the o&tp&t in a variabe so that e !an %o something &sef& ith it. For this, e &se su$process.check_output , hi!h ta-es ea!ty the same arg&ments as su$process.call: current/month B su-process.chec:/output(*-indate C[N*, shellB)rue#
J&st i-e hen rea%ing fie !ontents, the o&tp&t from an eterna program !an r&n over m&tipes ines that en% ith ne ine !hara!ters, so yo& probaby nee% to &se rstrip to remove them before !arrying o&t any pro!essing.
,ser input makes our programs more fle!ible $he eer!ises an% eampes that eve seen so far in this boo- have &se% to %ifferent ays of getting %ate into a program. For sma bits of %ata, i-e short D;4 se<&en!es, restri!tion enyme motifs, an% gene a!!ession names, eve simpy store% the %ata %ire!ty in a variabe i-e this: dna B *A)&%A)&%)%A&)A%&)A&%*
hen %ata is mie% in ith the !o%e in this manner, it is sai% to be har"Bco"e" . For arger pie!es of %ata, i-e onger D;4 se<&en!es an% sprea%sheeti-e %ata, eve typi!ay rea% the information from an eterna tet fie. For many p&rposes, this is a better so&tion than har%!o%ing the %ata, as it aos the separation of
202
Chapter @: Fies, programs, an% &ser inp&t
%ata an% !o%e, ma-ing o&r programs easier to rea%. 7oever, in a the eampes eve seen so far, the names of the fies from hi!h the %ata are rea% are sti har% !o%e%. (oth of these approa!hes to getting %ata in to o&r program have the same short!omings 5 if e ant to !hange the inp&t %ata, e have to open &p the !o%e an% e%it it. 'n the !ase of har%!o%e% variabes, e have to e%it the statement here the variabes are !reate%. 'n the !ase of fies, e have to !hoi!es 5 e !an either e%it the !ontents of the fie, or e%it the har%!o%e% fie name. eaife &sef& programs %ont generay or- that ay. 'nstea%, they generay ao &s to spe!ify inp&t fies an% options at the time hen e r&n the program, rather than hen ere riting it. $his aos programs to be m&!h more feibe an% easier to &se, espe!iay for a person ho %i%nt rite the !o%e in the first pa!e. 'n the net !o&pe of se!tions ere going to see a !o&pe of toos for getting &ser inp&t, b&t more importanty ere going to ta- abo&t the transition from riting a program thats ony &sef& to yo&, to riting one that !an be &se% by other peope. $his invoves starting to thin- abo&t the eperien!e of &sing a program from the perspe!tive of a &ser. $here are many reasons hy yo& might nee% yo&r programs to be &sabe by somebo%y hos not famiiar ith the !o%e. 'f yo& rite a program that soves a probem for yo&, !han!es are that it !o&% sove a probem for yo&r !oeag&es an% !oaborators as e. 'f yo& rite a program that forms a signifi!ant part of a pie!e of or- hi!h yo& ater ant to p&bish, yo& many have to ma-e s&re that hoever is peerrevieing yo&r paper !an get yo&r program or-ing as e. ?f !o&rse, ma-ing yo&r program easier to &se for other peope means that it i aso be easier to &se for yo&, a fe months after yo& have ritten it hen yo& have !ompetey forgotten ho the !o%e or-s8
203
Chapter @: Fies, programs, an% &ser inp&t
nteractive user input $o get intera!tive inp&t from the &ser in o&r programs, e !an &se the input f&n!tion. input ta-es a singe string arg&ment, hi!h is the prompt to be %ispaye% to the &ser, an% ret&rns the va&e type% in as a string: accession B input(*9nter the accession name*# 4 do something ith the accession aria-le
$he input f&n!tion behaves a itte %ifferenty to other f&n!tions an% metho%s eve seen, be!a&se it has to ait for something to happen before it !an ret&rn a va&e 5 the &ser has to type in a string an% press enter. $he &ser inp&t i be ret&rne% as a string so if e nee% to &se is as something ese 5 e.g. a n&mber 5 e have to %o the !onversion man&ay= an% i en% ith a ne ine so e might ant to &se rstrip to remove it=. Capt&ring &ser inp&t in this ay re<&ires &s to thin- <&ite !aref&y abo&t ho o&r program behaves. #rograms that e rite to !arry o&t anaysis of arge %atasets i often ta-e a !onsi%erabe amo&nt of time to r&n, so its important that e minimie the !han!es of the &ser having to rer&n them. hen &sing the input f&n!tion, there are to sit&ations in parti!&ar that e ant to avoi%. ?ne is the sit&ation here e have a ongr&nning program that re<&ires some &ser inp&t, b&t %oesnt ma-e this fa!t !ear to the &ser. hat !an happen in this s!enario is that the &ser starts the program r&nning an% then sit!hes their attention to something ese, ass&ming that the program i !ontin&e to ma-e progress in the ba!-gro&n%. 'f the &ser %oesnt noti!e or is not at their !omp&ter= hen the program rea!hes the point here it re<&ires inp&t an% hats, the program may be st&!- aiting for inp&t for a ong time. $he other s!enario to avoi% is that here a program r&ns for some time before as-ing the &ser for inp&t, then fais to or- %&e to an in!orre!t inp&t or typo, re<&iring the &ser to restart the program from s!rat!h.
20B
Chapter @: Fies, programs, an% &ser inp&t
4 goo% ay to avoi% both of these probems is to %esign o&r programs s&!h that they !oe!t a ne!essary &ser inp&t at the start, before any ongr&nning tas-s are !arrie% o&t. e !an aso re%&!e the !han!es of in!orre!t inp&t on the part of the &ser by offering !ear instr&!tions an% %o!&mentation. 4n important part of &ser inp&t is input vali"ation 5 !he!-ing that the inp&t s&ppie% by the &ser ma-es sense. For eampe, yo& might re<&ire that a parti!&ar inp&t is a n&mber beteen some minim&m an% maim&m va&es, or that its a D;4 se<&en!e itho&t ambig&o&s bases, or that its the name of a fie that m&st eist. 4 goo% strategy for inp&t vai%ation is to !he!- the inp&t as soon as its re!eive%, an% give the &ser a se!on% !han!e to enter their inp&t if its fo&n% to be invai%. ?ne big %raba!- of getting &ser inp&t intera!tivey is that it ma-es it har%er to r&n a program &ns&pervise% as part of a or- fo. For most bioogi!a anayses, spe!ifying program options hen its r&n &sing !omman% ine arg&ments is a better approa!h.
4omman" line arguments 'f yo&re &se% to &sing eisting programs that have a !omman%ine &ser interfa!e as oppose% to a graphi!a one= then yo&re probaby famiiar ith !omman% ine arg&ments1. $hese are the strings that yo& type on the !omman% ine after the name of a program yo& ant to r&n: myprogram one to three
'n the above !o%e, one to an% three are the !omman% ine options. $o &se !omman% ine arg&ments in o&r #ython s!ripts, e import the sys mo%&e. e !an then a!!ess the !omman% ine arg&ments by &sing the spe!ia ist sys.arg. &nning the fooing !o%e:
1
;ot to be !onf&se% ith the arg&ments that e give to f&n!tions, atho&gh they %o a simiar 9ob.
20
Chapter @: Fies, programs, an% &ser inp&t
import sys print(sys.arg#
ith the !omman% ine: python myprogram.py one to three
shos ho the eements of sys.arg are ma%e &p of the arg&ments given on the !omman% ine: $myprogram.py, one, to, three
;ote that the first eement of sys.arg is aays the name of the program itsef, so the first !omman% ine arg&ment is at in%e one, the se!on% at in%e to, et!. J&st i-e ith input, options an% fienames given on the !omman% ine are store% as strings, so if, for eampe, e ant to &se a !omman% ine arg&ment as a n&mber, e have to !onvert it ith int. Comman% ine arg&ments are a goo% ay of getting inp&t for yo&r #ython programs for a n&mber of reasons. 4 the %ata yo&r program nee%s i be present at the start of yo&r program, so yo& !an %o any ne!essary inp&t vai%ation i-e !he!-ing that fies are present= before starting any pro!essing. 4so, yo&r program i be abe to be r&n as part of a she s!ript, an% the options i appear in the &sers she history.
(ecap e starte% this !hapter by eamining to feat&res of #ython that ao yo&r programs to intera!t ith the operating system 5 fie manip&ation an% eterna pro!esses. e earne% hi!h f&n!tions to &se for !ommon fie system operations,
20G
Chapter @: Fies, programs, an% &ser inp&t
an% hi!h mo%&es they beong to. e aso ssa to ays to !a eterna programs from ithin yo&r #ython program. hen &sing these te!hni<&es to sove rea ife probems, or hen or-ing on the eer!ises, remember that yo& may en!o&nter errors that are nothing to %o ith yo&r program. For instan!e, hen trying to manip&ate fies yo& may get an error if a spe!ifie% fie %oesnt eist or yo& %ont have the ne!essary permissions to rename it. "imiary, if yo& get &nepe!te% o&tp&t hen r&nning an eterna program the probem may ie ith the eterna program or ith the ay that yo&re !aing it, rather than ith yo&r #ython program. $his is in !ontrast to the rest of the eer!ises in this boo-, hi!h are mosty sef!ontaine%. 'f yo& r&n into %iffi!&ties hen &sing the toos in this !hapter, !he!- the eterna fa!tors as e as !he!-ing yo&r program !o%e. 'n the ast portion of the !hapter, e sa to %ifferent ays to get &ser inp&t hen yo&r program r&ns. *sing !omman% ine arg&ments is generay better for the type of programming that forms part of s!ientifi! resear!h.
20H
Chapter @: Fies, programs, an% &ser inp&t
!ercises 'n the chapter. fo%er in the eer!ises %onoa% there is a !oe!tion of fies ith the etension ="na hi!h !ontain D;4 se<&en!es of varying ength, one per ine. *se this set of fies for both eer!ises.
>inning '.4 seuences rite a program hi!h !reates nine ne fo%ers 5 one for se<&en!es beteen 100 an% 1@@ bases ong, one for se<&en!es beteen 200 an% 2@@ bases ong, et!. rite o&t ea!h D;4 se<&en!e in the inp&t fies to a separate fie in the appropriate fo%er.
20I
Chapter @: Fies, programs, an% &ser inp&t
&olutions >inning '.4 seuences $he first 9ob is to fig&re o&t ho to rea% a the D;4 se<&en!es. e !an get a ist of a the fies in the fo%er by &sing os.listdir , b&t e have to be !aref& to ony rea% D;4 se<&en!es from fies that have the right fie name etension. 7eres a bit of !o%e to start off ith: import os for file/name in os.listdir(*.*#! if file/name.endsith(*.dna*#! print(*reading seuences from * C file/name#
e !an !he!- the o&tp&t to ma-e s&re that ere ony going to pro!ess the !orre!t fies: reading reading reading reading reading reading reading reading reading reading
seuences seuences seuences seuences seuences seuences seuences seuences seuences seuences
from from from from from from from from from from
xag.dna xaD.dna xaa.dna xa-.dna xai.dna xae.dna xah.dna xaf.dna xac.dna xad.dna
$he net step is to rea% the D;4 se<&en!es from ea!h fie. For ea!h fie that passes the name test, e open it, then pro!ess it one ine at a time an% !a!&ate the ength of the D;4 se<&en!e:
20@
Chapter @: Fies, programs, an% &ser inp&t
4 loo: at each file for file/name in os.listdir(*.*#! if file/name.endsith(*.dna*#! print(*reading seuences from * C file/name# dna/file B open(file/name# 4 loo: at each line for line in dna/file! dna B line.rstrip(*n*# length B len(dna# print(*found a dna seuence ith length * C str(length##
;oti!e ho eve &se% rstrip to remove the ne ine !hara!ter 5 e %ont ant to in!&%e it in the !o&nt of the se<&en!e ength, sin!e its not a base. ith ten fies, an% ten D;4 se<&en!es per fie, this program generates over a h&n%re% ines of o&tp&t 5 heres the first fe: reading found a found a found a found a found a found a found a found a found a found a reading found a found a found a
seuences from xag.dna dna seuence ith length dna seuence ith length dna seuence ith length dna seuence ith length dna seuence ith length dna seuence ith length dna seuence ith length dna seuence ith length dna seuence ith length dna seuence ith length seuences from xaD.dna dna seuence ith length dna seuence ith length dna seuence ith length
'32 G1G "0' G7J "1J 00 11J 3'1 303 '"J 121 ''2 20
$his oo-s goo% 5 ere getting a range of %ifferent sies. ;et e have to fig&re o&t hi!h bin ea!h of the se<&en!es sho&% go in. (e!a&se the imits of the bins foo a reg&ar pattern, e !an &se the range f&n!tion to generate them. e !an generate a ist of the oer imits for ea!h bin by ta-ing a range of n&mbers from
210
Chapter @: Fies, programs, an% &ser inp&t
100 to 1000 ith a step sie of 100, then a%%ing @@ to get the &pper imit of the bin. e go thro&gh this pro!ess for ea!h se<&en!e, !he!-ing if it beongs in ea!h bin in t&rn: 4 go through each file in the folder for file/name in os.listdir(*.*#! 4 chec: if it ends ith .dna if file/name.endsith(*.dna*#! print(*reading seuences from * C file/name# 4 open the file and process each line dna/file B open(file/name# for line in dna/file! 4 calculate the seuence length dna B line.rstrip(*n*# length B len(dna# print(*seuence length is * C str(length## 4 go through each -in and chec: if the seuence -elongs in it for -in/loer in range(100,1000,100#! -in/upper B -in/loer C JJ if length +B -in/loer and length ; -in/upper! print(*-in is * C str(-in/loer# C * to * C str(-in/upper##
$here are <&ite a fe eves of in%entation in the above !o%e, so yo& might have to rea% it thro&gh a fe times. e have •
the oop for ea!h fie name
•
the if statement that !he!-s the fie name
•
the oop for ea!h se<&en!e in a fie
•
the oop for ea!h bin
•
the if statement that !he!-s if the se<&en!e beongs in the bin
211
Chapter @: Fies, programs, an% &ser inp&t
$he first fe ines of the o&tp&t sho that this approa!h or-s: reading seuences from xag.dna seuence length is '32 -in is '00 to 'JJ seuence length is G1G -in is G00 to GJJ seuence length is "0' -in is "00 to "JJ seuence length is G7J -in is G00 to GJJ seuence length is "1J -in is "00 to "JJ seuence length is 00 -in is 00 to JJ seuence length is 11J
$he fina step is to !reate the ne fo%ers, an% rite ea!h D;4 se<&en!e to the appropriate one. e !an re&se o&r range i%ea to generate the fo%er names an% !reate them. $he name of the fo%er for a given bin is the oer imit, fooe% by an &n%ers!ore, fooe% by the &pper imit: for -in/loer in range(100,1000,100#! -in/upper B -in/loer C JJ -in/folder/name B str(-in/loer# C */* C str(-in/upper# os.m:dir(-in/folder/name#
hen e ant to rite o&t D;4 se<&en!e to a fie in a parti!&ar fo%er, e !an &se the same naming s!heme to or- o&t the name of the fo%er. ?f !o&rse, e aso have to fig&re o&t hat to !a the in%ivi%&a fies of D;4 se<&en!es. $he eer!ise %es!ription %i%nt spe!ify any -in% of naming s!heme, so e -eep things simpe an% store the first D;4 se<&en!e in a fie !ae% 1="na, the se!on% in a fie !ae% 2="na, et!. e nee% to !reate an etra variabe to ho% the n&mber of D;4 se<&en!es eve seen, an% to in!rement it after riting ea!h D;4 se<&en!e. 7eres the hoe s!ript 5 its by far the argest program that eve ritten so far:
212
Chapter @: Fies, programs, an% &ser inp&t
import os 4 create a ne folder for each -in for -in/loer in range(100,1000,100#! -in/upper B -in/loer C JJ -in/folder/name B str(-in/loer# C */* C str(-in/upper# os.m:dir(-in/folder/name# 4 create a aria-le to hold the seuence num-er se/num-er B 1 4 process all files that end in .dna for file/name in os.listdir(*.*#! if file/name.endsith(*.dna*#! print(*reading seuences from * C file/name# dna/file B open(file/name# 4 for each line, calculate the seuence length for line in dna/file! dna B line.rstrip(*n*# length B len(dna# print(*seuence length is * C str(length## 4 figure out hich -in the seuence -elongs in for -in/loer in range(100,1000,100#! -in/upper B -in/loer C JJ if length +B -in/loer and length ; -in/upper! 4 once e :no the correct -in, rite out the seuence print(*-in is * C str(-in/loer# C * to * C str(-in/upper## -in/folder/name B str(-in/loer# C */* C str(-in/upper# output/path B -in/folder/name C C str(se/num-er# C .dna output B open(output/path, **# output.rite(dna# output.close(# 4 increment the seuence num-er se/num-er B se/num-erC1
213
Chapter @: Fies, programs, an% &ser inp&t
$he tri!-y bit is fig&ring o&t the arg&ments to the range f&n!tion. e -no that e ant to start at ero an% in!rease by one ea!h time. $he finish position is the ength of the se<&en!e, min&s the -mer sie to ma-e s&re there is one -mers
21B
Chapter @: Fies, programs, an% &ser inp&t
orth of bases after it= min&s one to ao for the fa!t that the finish position is e!&sive=. $he range f&n!tion generates the start positions for ea!h -mer, an% to get the en% positions e 9&st a%% the -mer sie. e !an eamine the o&tp&t from this !o%e an% !he!- that it agrees ith o&t int&ition: A&)%)A%&)%)A&%)A%& A&)% &)%) )%)A %)A% )A%& A%&) %&)% &)%) )%)A %)A& )A&% A&%) &%)A %)A% )A%&
$o ma-e it easier to test this bit of !o%e, e t&rn it into a f&n!tion. $he f&n!tion i ta-e to arg&ments. $he first arg&ment i be the D;4 se<&en!e as a string, an% the se!on% arg&ment i be the -mer sie as a n&mber. 'nstea% of printing the ist of -mers, it i ret&rn a ist of them. 7eres the !o%e for the f&n!tion an% three statements to test it:
21
Chapter @: Fies, programs, an% &ser inp&t
def split/dna(dna, :mer/siEe#! :mers B $ for start in range(0,len(dna#F(:mer/siEeF1#,1#! :mer B dna$start!startC:mer/siEe :mers.append(:mer# return :mers
print(split/dna(*AA)%&)%&A)*, '## print(split/dna(*AA)%&)%&A)*, ## print(split/dna(*AA)%&)%&A)*, "##
4s e !an see from the o&tp&t, r&nning the f&n!tion m&tipe times ith the same D;4 se<&en!e b&t %ifferent -mer engths gives %ifferent res&ts, as epe!te%: $AA)%, A)%&, )%&), %&)%, &)%&, )%&A, %&A) $AA)%&, A)%&), )%&)%, %&)%&, &)%&A, )%&A) $AA)%&), A)%&)%, )%&)%&, %&)%&A, &)%&A)
;o e !an p&t this f&n!tion together ith the !o%e e %eveope% for ooping thro&gh fies from the previo&s eer!ise. $o !o&nt &p the -mers, e i !reate an empty %i!tionary at the start of the program ine 11=, then for ea!h -mer e fin%, e i oo- &p the !&rrent !o&nt for it in the %i!tionary ine 1@=. 'f the -mer is not fo&n% in the %i!tionary i.e. this is the first time eve seen that parti!&ar - mer= then e i say that the !&rrent !o&nt is ero. e then a%% one to the !&rrent !o&nt ine 20= an% store the res&t ba!- in the %i!tionary ine 21=.
21G
Chapter @: Fies, programs, an% &ser inp&t
1
import os 2 :mer/siEe B " 3 4 5 6 7 8 9
def split/dna(dna, :mer/siEe#! :mers B $ for start in range(0,len(dna#F(:mer/siEeF1#,1#! :mer B dna$start!startC:mer/siEe :mers.append(:mer# return :mers
10 11 12 13 14 15 16 17 18 19 20 21
:mer/counts B UV for file/name in os.listdir(*.*#! if file/name.endsith(*.dna*#! print(*reading seuences from * C file/name# dna/file B open(file/name# for line in dna/file! dna B line.rstrip(*n*# for :mer in split/dna(dna, :mer/siEe#! current/count B :mer/counts.get(:mer, 0# ne/count B current/count C 1 :mer/counts$:mer B ne/count
22 23
print(:mer/counts#
$his program generates a ot of o&tp&t8 7ere are the first fe ines so e !an see that its or-ing: Ugcagag! 11, aaataa! 13, ctttag! 11, gcagac! 1', ctttaa! 12, gcagaa! 1 etc. etc.
4s panne%, e en% &p ith a big %i!tionary here the -eys are -mers an% the va&es are their !o&nts. ;et, e have to pro!ess the kmer_counts %i!tionary. e go thro&gh the items in a oop, an% if the !o&nt is greater than some !&toff, e print the !o&nt. For testing, e fi the !&toff at 23 ater on e ma-e this a !omman%ine option=. 7eres the !o%e to pro!ess the %i!tionary:
21H
Chapter @: Fies, programs, an% &ser inp&t
count/cutoff B 23 for :mer, count in :mer/counts.items(#! if count + count/cutoff! print(:mer C * ! * C str(count##
4n% heres the o&tp&t e get: agagat agcggg atcgga aaggag cccagc aggttc agatta tctagg gagtgg ccggtt gagcag ttctga agatgg tctgaa gcgggt ttcaaa gattaa ccagcg ggacgt atggct
! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !
2" 2" 2 2 2' 2 2' 2' 2G 2" 2' 2" 2' 2' 2 2 2 2 27 2'
;eary %one. $he fina step is to repa!e the har%!o%e% va&es for the -mer sie an% the !o&nt !&toff ith va&es rea% from the !omman% ine. e 9&st have to import the sys mo%&e, an% !onvert the arg&ments to n&mbers &sing the int f&n!tion. 4s spe!ifie% in the eer!ise %es!ription, the first !omman% ine arg&ment is the -mer sie an% the se!on% is the !&toff. 7eres the fina !o%e ith !omments:
21I
Chapter @: Fies, programs, an% &ser inp&t
import os import sys 4 conert command line arguments to aria-les :mer/siEe B int(sys.arg$1# count/cutoff B int(sys.arg$2# 4 define the function to split dna def split/dna(dna, :mer/siEe#! :mers B $ for start in range(0,len(dna#F(:mer/siEeF1#,1#! :mer B dna$start!startC:mer/siEe :mers.append(:mer# return :mers 4 create an empty dictionary to hold the counts :mer/counts B UV 4 process each file ith the right name for file/name in os.listdir(*.*#! if file/name.endsith(*.dna*#! dna/file B open(file/name# 4 process each @
