MySQL Client Server

®

MySQL Client-Server Applications with ® Visual FoxPro

Whil Hentzen

Hentzenwerke Hentzenwerke Publishing

Published by: Hentzenwerke Publishing 980 East Circle Drive Whitefish Bay WI 53217 USA Hentzenwerke Publishing books are available through booksellers and directly from the publisher. Contact Hentzenwerke Publishing at: 414.332.9876 414.332.9463 (fax) www.hentzenwerke.com [email protected] MySQL Client-Server Applications with Visual FoxPro By: Whil Hentzen Technical Editor: Ted Roche Copy Editor: Nicole Robbins-McNeish Cover Art: “Teaching Tricks” by Todd Gnacinski, Milwaukee, WI Copyright © 2007 by Whil Hentzen All other products and services identified throughout this book are trademarks or registered trademarks of their respective companies. They are used throughout this book in editorial fashion only and for the benefit of such companies. No such uses, or the use of any trade name, is intended to convey endorsement or other affiliation with this book. All rights reserved. No part of this book, or the ebook files available by download from Hentzenwerke Publishing, may be reproduced or transmitted in any form or by any means, electronic, mechanical photocopying, recording, or otherwise, without the prior written permission of the publisher, except that program listings and sample code files may be entered, stored and executed in a computer system. The information and material contained in this book are provided “as is,” without warranty of any kind, express or implied, including without limitation any warranty concerning the accuracy, adequacy, or completeness of such information or material or the results to be obtained from using such information or material. Neither Hentzenwerke Publishing nor the authors or editors shall be responsible for any claims attributable to errors, omissions, or other inaccuracies in the information or material contained in this book. In no event shall Hentzenwerke Publishing or the authors or editors be liable for direct, indirect, special, incidental, or consequential damages arising out of the use of such information or material. ISBN: 1-930919-70-0 Manufactured in the United States of America.

iii

Our Contract with You, The Reader In which we, the folks who make up Hentzenwerke Publishing, describe what you, the reader, can expect from this book and from us.

Hi there! I’ve been writing professionally (in other words, eventually getting a paycheck for my scribbles) since 1974, and writing about software development since 1992. As an author, I’ve worked with a half-dozen different publishers and corresponded with thousands of readers over the years. As a software developer and all-around geek, I’ve also acquired a library of more than 100 computer and software-related books. Thus, when I donned the publisher’s cap five years ago to produce the 1997 Developer’s ( and didn’t like) from publishers, what Guide, I had some pretty good ideas of what I liked (and readers liked and didn’t like, and what I, as a reader, liked and didn’t like. Now, with our new titles for 2007, we’re in our tenth season. (For those who are keeping track, the ’97 DevGuide was our first, albeit abbreviated, season, the batch of six “Essentials” for Visual FoxPro 6.0 in 1999 was our second, and we've been publishing every year since.) John Wooden, the famed UCLA basketball coach, posited that teams aren’t consistent; they’re always getting better—or worse. We’d like to get better… One of my goals for this season is to build a closer relationship with you, the reader. In order for us to do this, you’ve got to know what you should expect from us. •

•

•

•

•

You have the right to expect that your order will be processed quickly and correctly, and that your book will be delivered to you in new condition. You have the right to expect that the content of your book is technically accurate and up-to-date, that the explanations are clear, and that the layout is easy to read and follow without a lot of fluff or nonsense. You have the right to expect access to source code, errata, FAQs, and other information that’s relevant to the book via our Web site. You have the right to expect an electronic version of your printed book to be available via our Web site. You have the right to expect that, if you report errors to us, your report will be responded to promptly, and that the appropriate notice will be included in the errata and/or FAQs for the book.

Naturally, there are some limits that we bump up against. There are humans involved, and they make mistakes. A book of 500 pages contains, on average, 150,000 words and several

iv

megabytes of source code. It’s not possible to edit and re-edit multiple times to catch every last misspelling and typo, nor is it possible to test the source code on every permutation of development environment and operating system—and still price the book affordably. Once printed, bindings break, ink gets smeared, signatures get missed during binding. On the delivery side, Web sites go down, packages get lost in the mail. Nonetheless, we’ll make our best effort to correct these problems—once you let us know about them. In return, when you have a question or run into a problem, we ask that you first consult the errata and/or FAQs for your book on our Web site. If you don’t find the answer there, please e-mail us at [email protected] with as much information and detail as possible, including 1) the steps to reproduce the problem, 2) what happened, and 3) what you expected to happen, together with 4) any other relevant information. I’d like to stress that we need you to communicate questions and problems clearly. For example… •

•

“Your downloads don’t work” isn’t enough information for us to help you. “I get a 404 error when I click on the Download Source Code link on www.hentzenwerke.com/book/downloads.html” is something we can help you with. “The code in Chapter 10 caused an error” again isn’t enough information. “I performed the following steps to run the source code program DisplayTest.PRG in Chapter 10, and I received an error that said ‘Variable m.liCounter not found’” is something we can help you with.

We’ll do our best to get back to you within a couple of days, either with an answer or at least an acknowledgment that we’ve received your inquiry and that we’re working on it. On behalf of the authors, technical editors, copy editors, layout artists, graphical artists, indexers, and all the other folks who have worked to put this book in your hands, I’d like to thank you for purchasing this book, and I hope that it will prove to be a valuable addition to your technical library. Please let us know what you think about this book—we’re looking forward to hearing from you. As Groucho Marx once observed, “Outside of a dog, a book is a man’s best friend. Inside of a dog, it’s too dark to read.” Whil Hentzen Hentzenwerke Publishing July 2007

v

List of Chapters Chapter 1: Why Client-Server? Why VFP? Why MySQL? Chapter 2: Development and Deployment Scenarios Chapter 3: Installing MySQL on Windows Chapter 4: Installing MySQL on Linux Chapter 5: Configuration of Users and Hosts Chapter 6: Connecting VFP to MySQL Chapter 7: Configuring MySQL Chapter 8: The Interactive Use of MySQL Chapter 9: Under the Hood: Where MySQL Keeps Its Data Chapter 10: Creating Data Sets from Scratch Chapter 11: Populating a MySQL Database: LOAD DATA INFILE Chapter 12: Populating a MySQL Database Programmatically Chapter 13: Advanced Data Issues Chapter 14: Constructing SQL to Avoid SQL Injection Chapter 15: Religious Wars: Remote Views, CursorAdapters, and SQL PassThrough Chapter 16: A Client-Server State of Mind Chapter 17: xBase to SQL Conversion Issues Chapter 18: A Client-Server User Interface for Querying Chapter 19: A Client-Server User Interface for Add/Edit/Delete Chapter 20: Relational Integrity Chapter 21: Getting Started with Stored Procedures Chapter 22: Deployment

1 17 25 55 75 93 119 151 173 181 211 219 249 263 271 287 295 317 335 369 379 393

vi

vii

Table of Contents Our Contract with You, The Reader Acknowledgments About the Author How to Download the Files Introduction Chapter 1: Why Client-Server? Why VFP? Why MySQL What is Client-Server? Why Client-Server? Data access and security Backup Replication Clustering Stored procedures Referential integrity Scalability What is VFP? Why VFP? Technical reasons Business reasons What is MySQL? Why MySQL? Technical reasons Business reasons Arguments against FoxPro is dead (or obsolete, or not supported, or...) Visual FoxPro won’t have a native 64 bit version ODBC is too old/slow MySQL’s ‘SQL’ is not compliant or standard Conclusion/Summary

Chapter 2: Development and Deployment Scenarios The big picture of how pieces fit together The components Visual FoxPro ODBC Drivers MySQL The scenarios Development Production Conclusion/Summary

iii xvii xix xxi xxiii

1 2 3 3 3 3 3 3 4 4 4 5 5 8 9 9 10 11 12 12 13 13 14 15

17 17 17 18 19 19 20 20 23 24

viii

Chapter 3: Installing MySQL on Windows

25

Software and environment The big picture – what installation does The essential installation process Read the documentation Download files Decide on a target location Grab the files Verify the bits Uninstalling an old MySQL installation Stop the service Remove the service Install the MySQL files with the installer The Setup Wizard Instance Configuration Wizard Potential problems Firewall troubles Anti-virus conflicts Collisions with a prior installation Installation results MySQL files have been installed MySQL service created and started Conclusion/Summary

25 25 26 26 26 26 27 30 30 30 31 31 31 38 47 47 49 50 52 52 53 54

Chapter 4: Installing MySQL on Linux

55

Software and environment used The big picture – what installation does The essential installation process Read the documentation Download the files Decide on a download target Grab the files Verify the bits Uninstalling an old version Install the files Install the server Install the client Set a password for the MySQL root user Potential problems Firewall conflicts Installation with Fedora Core 5/6 Installation results MySQL files have been installed The Linux user account is created Data files are created

55 55 56 56 57 58 58 62 62 64 64 65 66 66 66 67 69 69 70 71

ix

Automatic startup entries are created The mysqld daemon is started Conclusion/Summary

Chapter 5: Configuration of Users and Hosts Work with the server from the command line Windows and Linux command line tools Running MySQL scripts Summing up The MySQL interactive environment (MySQL monitor) Enter the MySQL interactive environment Exit the MySQL interactive environment A quick tour of the MySQL monitor Set passwords and new accounts Quickly adding a user MySQL password concepts How MySQL user passwords affect commands Adding a work-a-day user Conclusion/Summary

Chapter 6: Connecting VFP to MySQL Installing the MySQL ODBC driver Uninstalling an old version of the MySQL ODBC driver Downloading the MySQL ODBC driver Installing the MyODBC driver Connecting to MySQL from VFP Connecting to MySQL from VFP with a DSN Connecting to MySQL from VFP with a connection string DSNs or connection strings? Initial tests with Visual FoxPro and MySQL Creating a database Where is the data created? Working with tables in our new database Conclusion/Summary

Chapter 7: Configuring MySQL The MySQL configuration file my.ini in Windows my.cnf in Linux Downloading the MySQL Administrator Installing the Administrator On Windows On Linux Running the Administrator Advanced login options

72 73 74

75 75 75 76 83 83 83 84 85 86 86 87 89 90 91

93 93 93 93 94 99 99 108 114 114 115 116 116 118

119 119 119 121 122 124 124 128 131 133

x

The Administrator Editing the my.ini/my.cnf file directly Specific settings in the MySQL configuration file Conclusion/Summary

Chapter 8: The Interactive Use of MySQL Installation of the MySQL Query Browser Windows download Linux download Windows installation Linux installation Using the Query Browser Starting the Query Browser Startup Errors Parts of the Query Browser Functions of the Query Browser Other Tools Conclusion/Summary

Chapter 9: Under the Hood: Where MySQL Keeps Its Data

140 147 148 149

151 151 151 152 152 156 156 156 159 161 167 170 171

173

Types of MySQL data Where is your data? Windows Linux Where is the MySQL metadata? MyISAM InnoDB Where is the MySQL privilege data? Moving your Windows data out of Program Files Conclusion/Summary

173 173 174 175 176 176 177 177 179 180

Chapter 10: Creating Data Sets from Scratch

181

Best practices in database design Relational database theory Naming conventions Keys Time stamps Overloading tables Choosing data types Available data types in MySQL Comparing MySQL data types to VFP's choices Making the decision Creating your database from scratch Creating a database Creating tables

181 181 182 183 184 184 185 186 189 190 190 191 192

xi

Field attributes Indexes Creating primary keys Assigning foreign keys Working with time stamps End result: empty data set – what's it look like? mysqlshow SQL commands Getting data into the database Conclusion/Summary

195 198 199 200 205 206 206 207 210 210

Chapter 11: Populating a MySQL Database: LOAD DATA INFILE 211 The MySQL LOAD DATA INFILE command Where do I run the command? LOAD INFILE Syntax Handling primary keys Leading spaces Date and date-time fields Handling BLOBS and TEXT fields Conclusion/Summary

Chapter 12: Populating a MySQL Database Programmatically Converting DBF data to MySQL An overview of the process Setting up the MySQL tables for this chapter Description of the sample database Sample source code A simple program skeleton The SQLEXEC command Formatting commands for SQLEXEC Converting to a string for SQLEXEC Using variables Handling multiple data types Tip: Getting the structure of an existing MySQL table A better way to format SQL commands for SQLEXEC Creating the sample MySQL database Drop the existing tables Create the tables A simple, one table migration Using SQL INSERT to copy data Putting the INSERT in context A bit of complexity Sample migration with multiple data types Dealing with special characters Dealing with dates Handling BIT fields

212 212 213 214 214 214 215 218

219 219 220 220 220 222 223 224 224 224 225 226 227 227 229 229 230 231 231 231 232 234 234 235 236

xii

Adding error trapping to the migration process Alert about the error Using AERROR() in a program What should the program do? Debugging the error Not all errors are bad! Error handling is just the start Reorganizing the migration code Handling SQL commands that are executed once Handling SQL commands that are executed multiple times Passing long strings Moving more than data Performance issues Conclusion/Summary

Chapter 13: Advanced Data Issues BIT field operations Strategic use of BIT fields to replace VFP logicals A bit arithmetic and functions primer Define a BIT field Putting data into a BIT field New functions for modularizing code Get data out of a BIT field To extract just one position To input just one position New functions for creating cursors When to use BLOB fields to store files Infrastructure Permanence and incremental backups File size Dealing with NULLs What is a NULL? Setting up a NULL ENUM fields Conclusion/Summary

Chapter 14: Constructing SQL to Avoid SQL Injection How SQL Injection works Improperly trapped delimiters Direct injection: failure to validate data Quoted injection: failure to trap data type Database engine defects Solutions to SQL Injection Validate the data Parameterized queries Additional defenses

236 236 237 239 240 241 242 243 243 244 246 246 247 248

249 249 249 250 252 253 253 254 256 257 257 258 258 258 259 259 259 260 261 261

263 263 264 265 266 266 266 266 267 269

xiii

Conclusion/Summary

Chapter 15: Religious Wars: Remote Views, CursorAdapters, and SQL PassThrough Remote Views How to use Pros Cons CursorAdapter How to use Pros Cons SQL PassThrough How to use Pros Cons Decision time Conclusion/Summary

Chapter 16: A Client-Server State of Mind A bird’s-eye view of a client-server application Startup Menu File card forms Action-oriented forms Connecting to the back end Selecting a subset of records Writing back to the database Adding Deleting Updating Closing the connection Conclusion/Summary

Chapter 17: xBase to SQL Conversion Issues Architecture Intermixed xBase SQL commands Central data access mechanism Built with views Command comparison SQL commands xBase commands Function cross reference Interactive evaluation of MySQL functions

270

271 271 271 282 282 283 283 284 284 285 285 285 285 285 286

287 288 288 288 288 290 290 290 291 291 291 291 292 293

295 295 295 295 295 297 297 297 299 299 300

xiv

Math functions Lists, comparison, logic functions Empty/Blank/NULL functions Data/database functions String functions Date functions System information functions Miscellaneous functions Conclusion/Summary

Chapter 18: A Client-Server User Interface for Querying The logical and physical components Source code Base classes Simple navigation form Simple navigation with children form Simple navigation with filter Simple navigation with MySQL back end Pass connection parameters to simplenav_withmysql1 Store connection handle in simplenav_withmysql2 Simplenav_withmysql2.init() CreateConnection() GetResults() TableToForm() Close() Where do you create connections? Trade-offs for child and minor entity relationships Child tables Minor entities Conclusion/Summary

301 303 305 305 307 310 314 315 315

317 317 319 319 320 321 322 325 326 326 326 327 327 329 330 331 332 333 333 334

Chapter 19: A Client-Server User Interface for Add/Edit/Delete 335 Creating a class library for client-server functions Building a simple for to add records User interface Internal design Saving data A simple two table edit form User interface Internal design Displaying data on the form Saving data A simple two table delete form Denormalized table issues The Delete() method An all-in-one form

335 336 336 337 338 339 340 341 341 343 345 346 347 350

xv

Design of an all-in-one form Internal design Adding and deleting minor entities Adding a business Deleting a business Adding and deleting business types Adding and deleting contacts Tricks and Tips Retrieving empty dates from MySQL Retrieving null dates from MySQL Handling full outer joins in MySQL Selecting number of records in a result set Using auto-increment to handle primary key Determining the last value of an auto-increment field Conclusion/Summary

Chapter 20: Relational Integrity Revisiting the interface Creating foreign key constraints Using foreign key constraints Using RI interactively Using RI in a VFP form Errors encountered during RI setup InnoDB only! Indexes needed on both side of FK expression Index fields need to be same type, and maybe length Constraint name needs to be unique Conclusion/Summary

350 352 354 354 355 357 360 362 362 363 363 365 365 366 367

369 369 371 374 374 375 375 376 376 376 377 377

Chapter 21: Getting Started with Stored Procedures

379

What is a stored procedure Creating stored procedures in MySQL Make sure MySQL is set up for stored procedures Create your very own stored procedure Run the proc from the MySQL monitor Create a procedure that accesses a database table A stored procedure with a parameter Passing a parameter, somewhat more involved Using a parameter more constructively Using input and output parameters Cleaning up Where is a stored procedure stored? Calling a MySQL stored procedure from VFP Calling your basic stored procedure from VFP Passing a hard-coded parameter to a stored procedure Passing a user-provided parameter to a stored procedure

379 379 379 381 382 383 383 384 384 385 385 387 387 388 389 389

xvi

Incorporating stored procedures in the sample app Conclusion/Summary

Chapter 22: Deployment Getting started Development Build your application’s EXE with VFP Create an installation package for your application Production server Install production MySQL server Configure the production MySQL server user Install production data Open firewall for port 3306 Client machine Install the application Test connection to the server Licensing Internal applications Consultant The GPL What next? Conclusion/Summary

390 391

393 393 394 394 394 395 395 395 397 400 400 400 400 401 401 402 402 403 403

xvii

Acknowledgments A book isn’t just the result of the author staying up late at night with the Megaslurp-size of Jolt Cola and a bucket of Doritos by his side. Many folks are involved with the production of the book, from the initial formulation of the content all the way through the last application of spit and polish on the final manuscript. This book, even more so. I’ve been writing books since, oh, before there was paper, it seems. (1992 is the actual year, if you want to Magellan it on the Web). In those days, the process for writing a book adhered to a well-traveled formula. The author wrote a chapter, sent it to the publisher, who forwarded it to the technical editor, who returned it to the publisher, and back to the author, who had already been hard at work on the next chapter. Once the review of tech-edits was complete, the process was repeated, but with the chapter going back to the publisher for copy edit instead. And then the cycle was repeated again for layout, and then page proofs. This cycle kept the author on their toes and ensured that the book would be produced on time, but tended to drive most of the folks in the chain crazy – despite an initial outline that helped the author win the gig, the author often didn’t authoritatively know what Chapter N was going to contain until Chapter N+1 was drafted. And let’s not get started on trying to produce content about a product that was still in early beta testing. I used to have hair, before trying to write a book on Visual FoxPro 3.0. One big part of the book writing cycle was that the existence of the book was a secret. Oh, sometimes an author would let on that they were working on ‘something’, and as the software (and book publishing) industry matured, general topics became known. But details, no, they were kept behind closed doors, very hush-hush, eyes-only, and all that. We kind of broke that mold when we started offering pre-release chapters of our early Visual FoxPro 6.0 books in 1998, so people could get their hands on material before the book was finished. That proved popular, although more of an administrative headache than we planned on. Still, after a dearth of VFP books in the mid-90s, any new content was gladly welcomed, no matter how rough it was at first. Then the Internet happened, and with it, new expectations for publishing. One upside of those new expectations was collaborative development of documentation. So instead of trying to write this book in a closet and then release it to an unsuspecting world when I deemed it complete, I opened up the writing process, similar to the way Tom Rettig provided access to his third party framework, Tom Rettig’s Office, while it was under development. Over a hundred folks answered the initial call, and despite a couple of time spans where other projects slowed down progress on this book, a couple dozen stuck with me for a significant part of the gig. In no particular order, the following folks suffered with me through 2006 and beyond, and contributed something notable toward this book: Vassilis Aggelakos, Larry Bradley, Joel Chaban, Joe Cross, Hernan E. Delgado, Ray Del Prado, Bill Ganger, Matt Jarvis, Chris Jefferies, Carl Karsten, Bob Klein, Tony Kovalik, Bob Lee, David Masri, Brent Mattox, Ed Pecyna, Eric Selje, Bill Sherman, Phil Sherwood, Ken Srigley, Arden Weiss, and Bud Wheeler.

xviii

Oh, wait, that’s alphabetical order, isn’t it? Still, every person contributed something special for this book, and I thank each of them. But, wait, there’s more! I want to mention the specific contributions of... Dave Bartkus and Iames Pizzoli were as particular as reviewers as I’ve run across, constantly finding missing words, typos, and misspelled words. It eventually became a challenge to produce a chapter that either wouldn’t return full of edits. (Never succeeded.) Ron Gafron asked many real world questions, making sure I wasn’t getting bogged down in the details and missing the larger view for someone new to client-server and MySQL. Andy Kramek passed along many tidbits of client-server wisdom; his years of building big applications showed through regularly. Pat Mazuel steadily worked through every chapter, despite work, travel, and holiday commitments, acting as the architypical developer-reader I’d hoped for. Randy Pearson gave me a couple of well-needed slaps upside the head about SQL Injection. Nicole Robbins-McNeish, my copy-editor, proof-reader, and layout artist, who again turned what I thought was pretty good prose into something measurably better, and Todd Gnacinski, the hand behind the artwork of all but four Hentzenwerke Publishing covers (the exceptions being HackFox 6 & 7, and the first two DevGuides) – every book is another reason to see what Todd’s got up his sleeve. And last, but certainly not least… Ted Roche, whose contribution can’t adequately be described in words. Ted and I met on a bus on the way to Martin Schiff’s pre-DevCon party in Florida in 1992, and nary a week has gone by since without an exchange between us. Ted has been that perfect reviewer, some would say ‘coach’, alternating “Atta-boys” with “Are you sure about this?” and “Who are you trying to con?” and “Nice try, now go back and try again - the right way.” As I said in a recent email, “I open every chapter I get from you with trepidation, relieved at – while simultaneously dreading – your refusal to cut me any slack.” Thanks, dude. And finally, this time around on the “NP” list – the B-52s, Meat Loaf, Aerosmith and B.B.King. Looking back at the acks for my first book, I see things haven’t changed that much. You know what they say about writing software. Whil Hentzen Milwaukee, WI

xix

About the Author Whil Hentzen Whil Hentzen fills the role of “player-coach” at Hentzenwerke Publishing, a 9 year old Milwaukee-based publisher that specializes in titles covering high-end software development and, more recently, the migration of Windows developers and users to Linux. Whil started out life as a custom software developer using dBASE II (he still has the original 8 1/2 x 11 grey binder of documentation, much to the chagrin of his wife), and switched to FoxPro in 1990. As an independent Fox developer, he billed over 15,000 hours in the 90’s, including the requisite “1,000 hours in 3 months” Y2K emergency gig while recovering from leg surgery. During that time, he presented more than 70 papers at conferences throughout North America and Europe, edited FoxTalk, Pinnacle Publishing’s high end technical journal for 7 years, and hosted the Great Lakes Great Database Workshop since 1994. In 1999, Microsoft contracted with Whil to co-author the Certification Exam for Visual FoxPro 6.0 Distributed Applications, which has since been taken by thousands of developers to help them achieve Microsoft Certified Solution Developer credentials. He was a Microsoft Most Valuable Professional from 1995 through 2003 for his contributions to the FoxPro development community, and received the first Microsoft Lifetime Achievement Award for Visual FoxPro in 2001. He started writing in order to develop credentials to bolster his software development career. His first book, Rapid Application Development with FoxPro, showed up in 1993, and seven more have appeared since. He started Hentzenwerke Publishing in 1996 to self-publish a specialized volume on software development, and then began producing Visual FoxPro books when the major publishers abandoned the market in 1998. Sensing the impending acceptance of Linux on the desktop and anticipating a coming demand for custom business applications on those Linux desktops, he turned his attentions to Linux in 2002, and now has eight narrowly targeted Linux and open source books available. Despite being HQ’d in Whil’s house, HWP’s backlist now numbers close to 40, with some 70 authors and editors worldwide on the payroll. He currently splits his time between freelance custom development with Visual FoxPro and a host of open source tools such as MySQL, PHP, and Python, and running the publishing side of the biz. Whil has a B.S. in Mechanical Engineering from Rose-Hulman Institute of Technology, named as the United States’ top independent engineering school every year since 1999 by U.S. News & World Report. He served on RHIT’s Commission on the Future from 1993 to 1998. He currently spends his copious amounts of spare time with his five kids and volunteering for the local school district. An avid distance runner, he has logged over 50,000 miles lifetime, and, pending recovery from yet another injury-du-jour, hopes for one more shot at a competitive 5,000-meter clocking before age and common sense close the door on that activity. You can reach Whil at [email protected] or at 414.332.9876.

xx

Ted Roche Ted Roche is the owner of Ted Roche & Associates, a software development and consulting company based in Contoocook, New Hampshire, USA. TR&A develops and maintains database-intensive web sites and rich-client applications for clients in a range of industries from wholesale commodities to produce to financial services, manufacturing, and insurance. A former nine-time Microsoft Most Valuable Professional, Ted now spends his time developing Free / Open Source solutions using Linux, Apache, MySQL, PostgreSQL, PHP, Python, and Ruby. If you’d like to contact Ted for help on your project, his contact information can be found at http://www.tedroche.com In his spare time, Ted is the leader of the Central New Hampshire Linux User group and a board member of the Greater New Hampshire Linux User Group, http://www.gnhlug.org.

xxi

How to Download the Files Hentzenwerke Publishing generally provides two sets of files to accompany its books. The first is the source code referenced throughout the text. Note that some books do not have source code; in those cases, a placeholder file is provided in lieu of the source code in order to alert you of the fact. The second is the e-book version (or versions) of the book. Depending on the book, we provide e-books in either the compiled HTML Help (.CHM) format, Adobe Acrobat (.PDF) format, or both. Here’s how to get them.

Both the source code and e-book file(s) are available for download from the Hentzenwerke Web site. In order to obtain them, follow these instructions: 1.

Point your Web browser to www.hentzenwerke.com.

2.

Look for the link that says “Download”

3.

A page describing the download process will appear. This page has two sections:

•

•

4.

Section 1: If you were issued a username/password directly from Hentzenwerke Publishing, you can enter them into this page. Section 2: If you did not receive a username/password from Hentzenwerke Publishing, don’t worry! Just enter your e-mail alias and look for the question about your book. Note that you’ll need your physical book when you answer the question.

A page that lists the hyperlinks for the appropriate downloads will appear.

Note that the e-book file(s) are covered by the same copyright laws as the printed book. Reproduction and/or distribution of these files is against the law. If you have questions or problems, the fastest way to get a response is to e-mail us at [email protected]

xxii

Introduction

xxiii

Introduction Where I tell you what you're going to get out of this book, and some housekeeping notes. Blah, blah, blah...

I built my first real-live client-server application as part of a Y2K emergency bailout for a customer. The customer had contacted me in late 1998 with the desire to merge a number of disparate systems, both home-grown and outsourced, into a single coherent application. The company’s growth had recently accelerated; the firm had gone from a single quonset hut-based headquarters to manufacturing plants overseas and branch offices throughout the States in just a couple of years. Their systems hadn’t kept pace. Naturally, with the spectre of Y2K looming, the first question I asked was whether they had any Y2K issues looming. “No.” they declared. “None whatsoever. We’ve been through every application we use and we’re in good shape. Take your time and do it right.” So we began design, merging just about every department’s stand-alone application into a single system that would mesh with their existing vertical market accounting and sales call management applications. Summer of 1999 arrived, prototypes were rolled out and accepted, and we were ready to start coding. Fall approached, and new versions of the accounting and call management apps were installed. SQL-Server based versions. “We know that you’ve already started the programming, but how hard would it be to convert your system to use SQL-Server as well?” they asked. This wasn’t really a question, of course. Okay, this was going to be interesting. They understood that there was going to be some additional time needed to retool the interface, and I was going to be putting in some extra hours putting into practice for someone else what I had to that point only used internally. But we all have to start somewhere. Progress continued, and I was confident enough to schedule leg surgery six weeks out, just before Thanksgiving. As the kids started discussing Halloween costumes, the meeting to install the first beta version was scheduled. I arrived to find everyone gathered around a conference table. “I know you were planning on showing us the first beta today, but we have something else we need to discuss.” My stomach twisted; I could imagine the worst..... They were going out of business, they decided they weren’t happy with my work, they ran out of money, they were moving their headquarters, a new management team had taken over the company, who knew what it was going to be? “Our inventory system is going to shut down in 78 days. It turns out it’s not Y2K compliant after all. We need to have your entire system up and running by holiday shutdown on December 23.” Nothing like a bit of pressure to help focus one’s mind, eh? I wrote a lot of remote views and tested performance with queries simultaneously hitting three separate SQL Servers that fall... much of it with my leg wrapped up; the pity factor with the customer as I hobbled in with crutches wearing off quicker than you’d imagine. With the aid of a lot of duct tape and bubble-gum (and an understanding customer when yet another late night short-circuited the brain-keyboard connection), we were able to stitch up enough of the system for them to flip the switch over holiday shutdown and go live. The rest of

xxiv

MySQL Client-Server Applications with Visual FoxPro

the winter was a lot of clean up work and patching the holes, but the system was solid by spring; enough that I was able to turn over the keys to the source code before summer. I mention this not only because it’s a great Y2K story, and not just because it explains why I never want to see another Remote View as long as I live, but because it goes to show that building a big client-server isn’t that much more difficult than the traditional LAN application that you’re already used to. As the saying goes, “After you understand the concepts, it’s just syntax.” And this book is about the concepts (together with some sample code to show you the syntax.) It’s the book I wish I had when I was starting out with client-server development a decade ago. A couple of notes about what’s inside before we get started: 1. First of all, please see “How to download the files” for instructions on how to grab files from our website. The source code consists of one big ZIP file that consists of separate ZIP files for each chapter. Check out the “README.TXT” files inside each chapter’s ZIP file for details. 2. The screen shots and URLs throughout this book were current as of the writing and/or printing. Depending on when you are reading this, they might have changed. (“Might have”? Of course they have!) 3. I’ve written a number of articles that you might find useful; they can be found under the “Resources” link on our website. Expect to see some on MySQL as follow-ups to this book. That’s all for now. Enjoy the ride.

Chapter 1: Why Client-Server? Why VFP? Why MySQL? 1

Chapter 1 Why Client-Server? Why VFP? Why MySQL? VFP and MySQL make an incredible combination for fat-client, client-server and Web applications. However, Microsoft’s many years of marketing neglect has relegated VFP to near invisibility. Thus, its capabilities are largely unknown and its selection as a development tool has become a questionable choice in the eyes of many uninformed or prejudiced users. And MySQL, despite its growing acceptance in many areas of the IT arena, is still a relative newcomer in terms of mind-share with many US users due to its open source roots and European origins. This chapter explains why VFP and MySQL is a compelling combination, and gives you, the developer, ammunition for those “discussions” with management.

Since this book covers both VFP and MySQL, it’s likely that some readers are coming from an ‘experienced in VFP, inexperienced in MySQL’ perspective while others are more familiar with MySQL and less so with VFP. And it’s probable that some are new to clientserver completely. As a result, after discussing client-server architecture in general, I’ll cover what each tool is and the benefits of using it.

This chapter provides a technical overview of client-server systems and both products as well as the business justification for all three – but for two very different audiences. First, I’m targeting folks who are new to one or more of these areas, giving them the big picture so that they may put the rest of the book in proper context. If you’re already experienced in one or more of these areas, you may be thinking that you can skip this chapter, but that may not be advisable. During the preview of this book, I received some feedback that indicated this chapter was unnecessary. Bud Wheeler of Visionpace summed it up nicely: “This is my least favorite chapter in every technical book. It always feels like something you have to get through before you actually get started. As the reader I want to skip this chapter, but then I never know if I am missing something important. I think you are preaching to the choir. The reader wouldn’t have the book if they weren’t already going to be using VFP and MySQL. The majority of this content could be presented as an appendix or in other chapters.” I thought about this for a while, and I can see the point of view. Frankly, I don’t like writing these types of chapters either – they’re more like marketing than technical, and if I wanted to be in marketing, well, you know the rest of that joke. But I decided to go ahead and do this chapter anyway. Sorry, Bud! Here’s why. First, as I mentioned in the abstract above, there are some folks who are new to the whole client-server thing, and they need an introduction to the topic that is tailored for VFP and MySQL. Next, some folks are likely to be experienced with one tool but not the other, and so still need some of the basic “What is and why?” information. For them, VFP or MySQL is a buzzword they’ve heard bandied about, but don’t know that much about. The purpose of this

2


chapter for them is to summarize what they could dig up in an hour elsewhere. That’s what a book is, actually, isn’t it - a coherent synopsis of what you could find elsewhere if you had the time? Finally, while you and I and the guy behind the door all know what marvelous tools VFP and MySQL are, not everyone is as blessed. More importantly, while you may know in your heart that you’ve made the right choice, you may not be able to succinctly convey that knowledge and feeling to others. Many of you will end up in a meeting and have to justify your choice of VFP (“I thought Microsoft killed it a long time ago!”) or MySQL (“MySQL? Isn’t that one of those commie ‘open source’ things?”). It’s easy to forget items if you’re trying to come up with a presentation pres entation for ‘the big meeting with management’ tomorrow morning. This chapter provides that presentation. Here is where you’ll find a (moderately) objective list of benefits to using VFP, MySQL, and the two of them together. This is the chapter you take into your boss as backup to buttress your case for your choice of tools. If you’re just kicking the tires for now, or if you’ve already made the decision and don’t need any support for it, then feel free to move along to the next chapter. For the rest of you, let’s take a new look at two of our favorite development tools.

What is Client-Server? In a traditional file-server application (also called a LAN or fat-client application), the data is stored on the file server while the application itself runs on the user’s workstation. Because the workstation is used for processing, the application is used to provide sophisticated, powerful interfaces. The trade-off, however, is that the database is neither secure nor fault-tolerant. The file server is simply a repository for dumb files that happen to be database files, and thus are subject to the same types of malevolent access and inadvertent corruption as every other file on the server. In addition, something as simple as performing a query requires a lot of network overhead. Opening a table requires a hit to the file server. Then the index has to be opened and the keys determined – trip number two – possibly encompassing a lot of data if the indexes are large. Third, now that the index keys have been determined, the specific records requested need to be grabbed. That’s a lot of work and data going over the wire. If you have a lot of users and large databases, this can potentially bog down a network substantially. In a client-server application, the data is stored on the file server, but encapsulated inside an application so that only the application can access the data. The user (also called the ‘client’) simply sends a SQL statement to the database server, which then does all of the processing and returns the results – just the records that match the SQL statement. (In properly designed and configured systems, goofball scenarios – such as if a SQL statement requested every record in a multi-million record table – are handled appropriately.) As a result, the number of trips sent across the network is minimized, as is the amount of data. Thus, in a client-server system, two separate applications are involved. The first application – the database server – runs on a physical server box. It protects and provides access to the data, which is also located on the server box. The second application – the client app – runs on the user’s workstation, and sends requests to the server, just like clients at a restaurant make requests of the waiter/waitress (the ‘food server’).


Why Client-Server? As briefly discussed in the previous section, a client-server application can offer better performance than a corresponding file server application as well as better security. But there’s more. Let’s start at the beginning, though.

Data access and security Data is protected by the database server (and, presumably, by restricted access to the physical server box as well.) Clients send requests (like SQL SELECTS, INSERTS, UPDATES, or DELETES) to the server. The server application receives those requests, does whatever processing is necessary, produces a result set, and sends those results back to the client. Access to the database is controlled by the server. Users can’t get to the data unless the server application allows them to. Servers themselves have multiple layers of security built in – first allowing access to the server, and then to the data on the server. MySQL also has the ability to restrict access from predefined clients.

Backup Modern client-server databases provide the ability to do a live backup of the database while it is in use, which is difficult, if not impossible, to do with a file-system database like Visual FoxPro. While you can physically copy VFP files while they’re open, the duplicate files are subject to inconsistency even corruption if they’re being written to at the same the backup is taking place. With a client-server system, the engine takes care of writing an entirely separate copy of the database while making it consistent with writes from the user. The copy can then be moved to separate media, such as another server or tape.

Replication Many larger systems require the use of multiple copies of the same database. Replication is the process where the data in one database is used to update another, physically separate, database. Suppose a company has installations of the system in various locations around the globe, some of which have tenuous network connections. Instead of requiring each location to rely on a single master copy of the database “back at headquarters,” each location can have their own copy, which is then synchronized with the master copy overnight.

Clustering Clustering is the use of multiple systems to operate together, in order to spread the load among more than one box. The database file(s) can be located on more than one box, but the application acts as if they’re all in a single location. Many client-server systems provide clustering as a native feature, while accomplishing the same task in Visual FoxPro would require significant amounts of effort.

Stored procedures Stored procedures are pieces of code contained with the database engine, and can either be called explicitly or executed by the database server engine as a result of an engine process. An example of the latter is the insertion of a new record. Stored procedures were commonly used for generating primary keys and handling child records when parents were deleted, until those functions were embedded directly in the engine itself. Some organizations require virtually all

4


access to the database to be done via stored procedures, using external applications only for querying and reporting. Properly written stored procedures can be valuable tools, as they move the processing pr ocessing demand to the presumably powerful server box instead of relying on a network and often widely disparate processing power on user workstations. Many database servers also allow stored procedures to be given security rights just like other objects in the database. Users can then only modify the database by going through the appropriately vetted stored procedure that is the only object that can touch the data.

Referential integrity Referential integrity (“RI”) is the set of rules defined for handling changes, including but not limited to deletion, that could affect records related to the one actually being changed. RI can be handled several ways. One way is to depend on stored procedures executed by the insert, update, or delete of a record (known as triggers). Another way is to build RI into the database at the engine level (known as “declarative RI”), so records with linked records are simply not allowed to be deleted or modified in such a way that would leave the linked records with incorrect relationships. File based database systems like Visual FoxPro provide RI through the use of triggers while database servers like MySQL use declarative RI. The problem with triggers is their stored procedures are built by human programmers on a custom basis, which renders them more fallible than a mechanism built into the server, and thus, presumably, tested more fully by the company that manufactures the server.

Scalability Client-server systems, by their very nature, are designed to handle hundreds, even thousands of users, and provide access to terabytes of data. With features like clustering and replication, client-server systems can natively handle the large majority of applications requiring large numbers of transactions with real-time response. There are other advantages to client-server architecture as well, but these are the major points you would be well advised to look at if you’re considering a move from a file-based architecture to a client-server setup.

What is VFP? What is Visual FoxPro? Why, it’s a floor wax! No, it’s a dessert topping! No, it’s both! With that required remembrance from a Saturday Night Live skit of many moons ago out of the way, we can concentrate on what VFP is. Visual FoxPro is an application development tool that incorporates both a powerful and full-featured programming language and a native data engine. The language has its roots in the original end-user database manipulation program, dBASE II, from the early 1980s, and thus is tightly integrated with data handling. The major overhaul of the tool in 1994 that produced Visual FoxPro from FoxPro for DOS and FoxPro for Windows incorporated a new objectoriented syntax that allowed backward compatibility with older systems while allowing stateof-the-art development as well. VFP’s predecessor, FoxBASE, has long been used for building single user database applications since the mid 80s. FoxBASE morphed into FoxPro, running on DOS, Windows, Mac, and Unix around 1990, and became the tool of choice for building multi-user systems on


local area networks. Indeed, many of those systems written 15 years ago are still around today, faithfully serving their business needs without a hiccup. Visual FoxPro became the perfect front end for client-server systems with native language extensions in the early 90s. Finally, with third-party add-ons like Web Connection, VFP provided an excellent way to add Web functionality to a system in the late 90s. During the 90s, VFP was used for a number of well known, high profile applications: CBS (Christian Broadcasting Company), based in Virginia, developed a system that stored 10,000,000 records in FoxPro for DOS. In 2007, that may not seem like a big deal, but in 1992, when the work-a-day machine sported a 386 processor and two to four megabytes of RAM, and was connected to a network via a 10 Mb card, this was an impressive feat. The Joint Flow and Analysis System for Transportation (JFAST) determines a variety of logistical modeling, including transportation requirements, source of action analysis, and delivery profile development, for the United States Military. Originally built on an IBM 370 mainframe computer, the development team turned to FoxPro when the 370 couldn’t perform fast enough. It produces reports that brief the Joint Chiefs of Staff during military operations, and has been personally demonstrated as an example of what is possible with Fox to industry executives like Fox Software founder Dave Fulton and Microsoft Chairman Bill Gates. The largest amount of data handled by a Fox application is 128 GB, the engineering database for the EuroTunnel underwater passageway between England and France. Surplus Direct, a mail-order retailer of computer hardware and software, began its Web appearance using Visual FoxPro and Web Connection. At its peak, VFP has handled as many as a million requests in a single day. In short, VFP can (darn nearly) do it all.

Why VFP? Visual FoxPro (and its predecessors) has been the most powerful, fastest desktop development tool for nigh onto 20 years. The world has changed in that time, with Web-enabled and mobile applications becoming more and more important. However, the need for rich user interfaces that work with network-level databases hasn’t gone away – we’ve just gotten used to them, so they’re not in the news much anymore. Let’s take a fresh look at the benefits VFP has to offer.

Technical reasons I’ve been developing database applications for local area networks, client-server systems, and Web systems for over two decades. Over the years, I’ve poked under the rocks of a half dozen different tools, and Visual FoxPro is still my favorite. Why? It’s a great development tool, built by a small group of folks who used to be Fox developers themselves. Here are some of the big reasons it’s a great choice for building client-server apps with MySQL.

Robust programming language Some would say too robust, in that there are literally thousands of commands and functions. Fortunately, since the core engine has stayed the same for the last decade, the development team has been able to spend their time tweaking and enhancing the existing toolset, instead of debugging a whole new build every couple of versions.

6


Full-featured IDE We take the Command Window – the ability to execute one or more lines of code interactively, for granted, but not all tools have it, and some have just acquired it recently. There’s also a great debugger with five separate windows, and the data session window, providing a view into tables and cursors with a single click. Programming tools The coverage profiler, document view, and code reference tools – use some of them, use all of them – your choice, but they’re there as you need. Object-oriented programming facilities These include a Properties GUI, visual and non-visual class tools, class and object browser, and a class library component gallery. Extensibility Fox has always provided developer-accessible hooks in its development and run-time environments. You can hook into all of the developer tools, such as the form designer, menu builder, report writer, project manager, and others, as well as specify replacements for all of the programming tools just mentioned. For example, you can write a routine that automatically runs when you open a project in the Visual FoxPro project manager, or when you execute the build process. Another example is shown in Figure 1, where you see the pointers to various programming tools. You can write your own replacement and point VFP to use it instead.

Figure 1. 1. You can point to your own replacement for a variety of programming tools.


In addition, many of the native tools and components are written in Visual FoxPro itself, and the source code for all such components is provided. So, if you need alternative performance or functionality in one of those tools, you can rewrite it yourself.

Executable and COM component creation within IDE Some developer tools require you to shell out to another mechanism to do your application builds. This is all handled within VFP’s IDE, and it has hooks you can use to modify the process as needed for your own projects. Native data engine Fire up VFP and you have direct access to the databases and tables at your fingertips in the IDE – no need to have an external data source installed and connected. (And you have access to that data through the afore-mentioned Command Window.) While that may seem counterproductive to a book that discusses VFP with a back-end database, what it means is you can use this internal data processing in your applications. Grab a record set off the server, and then use the local data engine to do processing and simply fire the result back to the server, or the user. Data massage capability One of my reviewers pointed this out in a most fluent manner: “Visual FoxPro is still my favorite tool to get data massaged and migrated from one (or even better, multiple) sources to a target data environment. For example, moving data from DBFs, Excel, Access, text files, or whatever other format you have your data in – it’s a snap to move it to MySQL, MS SQL Server or Oracle. The main reason is that it has the best string manipulation functions of any of the other tools I’m aware of. “Combine this with the friendly Command Window, which lets you run single commands or groups of commands in interactive mode from either native DBFs or any ODBC-capable data source. It’s ridiculously easy to select data from a varied set of sources into one or more cursors, and then process them one record at a time and use SQL pass-through SQL INSERT statements to move the results into the back end.” If you have used other tools to migrate data from one place to another, you, too, know how powerful Fox is – much like the difference between pen-and-paper spreadsheets compared to that new-fangled electronic spreadsheet that Bricklin and Frankston invented in the late 70s. Rushmore technology Oh yeah, that. We often forget that VFP is still the fastest desktop database in the solar system. Introduced over a decade ago, this patented technology provides sub-second responses to queries against multi-million record tables. While that may not seem like a big deal in 2007, that speed could be obtained on run-of-the-mill laptops in the mid-90s. And Fox has only gotten faster. This means internal processing of data sets of virtually any size is instantaneous. ActiveX extensions and Web services VFP applications can include outside tools and functionality via ActiveX controls, just like Visual Basic and other ActiveX enabled tools. Fox can also develop and consume Web services as well as use SOAP, if you’re so inclined.

8


Processing speed VFP has long been heralded for its data access speed. But that’s not all it’s fast at. Text processing operations, like the kind you’d use to build HTML output, has been significantly enhanced over the years. You’ll see that iterating through a loop 10,000 times while building a string is done instantaneously – smaller jobs are finished even faster. Quite a remarkable achievement for a 4GL (Fourth Generation Language.) Stability Some tools are rewritten every couple of versions, either to make up for the problems that surfaced after the last rev’s ‘hurry up and get it out the door’ approach have come back to roost, or simply to incorporate new features that couldn’t be added any other way. VFP’s core engine has been around for over 15 years. As a result, the product is incredibly stable. Backward compatibility The Fox team has always taken great pains to make sure new versions didn’t break old code. This is part of the reason the language has become so huge (some say ‘bloated’, for which an argument can be made), but it also means the code base has been tested exhaustively. This list just touches the surface, but I think it hits some of the key points other technical folks would be interested in. See the following URL for a detailed list of features: http://msdn.microsoft.com/vfoxpro/productinfo/features/default.aspx

Business reasons I’m going to keep this one short, as management, unlike the techies who will be interested in the ‘technical reasons’ just covered, sometimes has a limited attention span. Two (or three) big points are about all you’ll have time for.

Long life span An app you build (or rewrite) today can be expected to have a lifespan of at least a decade. How many VB apps from ten years ago are still in production? Inexpensive to deploy On the Fox side, you build an app and deploy the EXE as many places as you want. Total dollars out of pocket (in terms of license fees back to Microsoft) to do so: $0. (This is one major reason why MSFT won’t promote VFP – there’s no financial incentive – “client access license fees” – to do so.) RAD VFP is the original Rapid Application Development toolkit, and has been that way for a long, long time. Building systems in VFP, if you have the requisite skill set, is fast. Really fast. Poke around on any VFP-related mailing list and you’ll find instance after instance of a Fox developer creating an application several times faster than the other team slated to do the same gig in another tool/language. But the converse doesn’t ever seem to happen. I have never seen Fox booted out of place for technical reasons – rather, the motive always ends up on marketing’s lap – “It’s not strategic” or “It’s not .NET.” You never see stories about how it took a Fox developer five


times as long to build a system as the guy next door using Visual Basic or Java or C++. Why spend nine months building an app that can be done in three?

Community support The average Visual FoxPro developer can truthfully be called zealous in terms of their passion for the product; some unkind folks use monikers such as ‘rabid’, ‘fanatical’, or ‘possessed’. Whatever the case, the end result is that this passion has translated into a large community of very helpful and knowledgeable users who are available on-line through electronic forums and mailing lists, and in person at conferences and user groups.

What is MySQL? MySQL is a database server that competes in the same general arena as other mid-market database back-ends, such as Microsoft SQL Server, PostgreSQL, and Sybase. (Don’t confuse these with the high-end, dedicated big-iron database engines like Informix, Oracle, and IBM’s DB2.) Developed by the company MySQL A.B., which was founded by two Swedes and a Finn, the company is based in Sweden and has a dozen offices in major countries throughout the world. MySQL comes in several flavors. The MySQL Pro Certified Server is designed for enterprise and mission critical database applications. The MySQL Community Edition is for open source developers who want to get started with MySQL. The MySQL Cluster version provides fault tolerant database clustering. This book will cover the Community Edition, as it is freely available under the open source GPL license. I’ll cover when you need to consider licensing the Pro edition in Chapter 22, Deployment. As with other database servers, MySQL encompasses more than just the database server. You can also download a variety of add-ons, such as the MySQL Administrator, which allows you to edit the MySQL configuration file graphically, a Query Browser, which is similar in purpose to Microsoft SQL Server’s Query Analyzer, and a Data Converter, which can help moving data from another database backend. MySQL claims to be the world’s most popular open source database server, and while I doubt they’ve gone out and tallied up how many of which server is running on every box in the world, there’s no argument that it’s hugely popular and can be found in every nook and cranny of the IT world. It’s as close to a corporate standard as there can be. Who is using MySQL? There are lots of case studies on the MySQL website here: http://www.mysql.com/why-mysql/case-studies/ While some of the companies aren’t exactly household names, plenty of others are, and they all serve to demonstrate the wide variety of systems relying on MySQL as their database.

Why MySQL? This question can be interpreted two ways. The first is, “Why use a back-end database server instead of relying on VFP tables?” The second is, “If you decide to use a backend database server, why use MySQL instead of (fill in the blank with the name of a competitor or three)?” The reasons to the first question generally fall under the technical category while the reasons to the second generally end up under the business reasons domain.

10


Technical reasons There are several classic reasons to move to a back-end database server from VFP.

Limitations to the DBF: size, security, stability The first that comes to mind is the size limit of Visual FoxPro tables. A single file (DBF, FPT, or CDX) can’t be larger than 2 gigabytes, due to the internal architecture of VFP. (Who decided back in the 1980s that we wouldn’t need to use tables over 2 GB in size? Undoubtedly the same folks who knew we’d never need more than 640K of RAM.) MySQL (and other database servers) have limits that boggle the mind, perhaps even more than those 2 GB boggled twenty years ago. Suffice it to say that if you are considering tables that will exceed MySQL’s limits, you are probably better off considering one of the truly big iron systems – Oracle or DB2. The second reason often cited is security. VFP tables are simply files on disk. If you have an ODBC driver, or the appropriate VFP-enabled application (like Word or Excel), you can open up one of those tables and cause all sorts of mayhem. A related reason is the reliability of the VFP data structure. While VFP goes to great lengths to protect the structure of the DBF file system, a table (or its related index and/or memo files) can be corrupted, causing data loss and much anguish. Database servers, because of the way they’re built and maintained, are inherently more reliable. MySQL specifics Getting more specific, MySQL 5.0 (finally!) has all of the features you’d expect from a modern database server: size, speed, functionality. Table sizes Depending on the operating system and file system used on the OS, MySQL will handle file sizes up to 16 terabytes for 32 bit machines and 8 million terabytes on 64 bit machines. That’s each file – not an entire database. See “1.4.4 How Large MySQL Tables Can Be” or http://dev.mysql.com/doc/refman/5.0/en/table-size.html for details. Speed MySQL can go up against the big boys for a variety of large applications, making it perfectly suitable for the small to midsize company and sub-enterprise level applications that VFP has proved its chops at handling. DBAs like to look at the raw numbers as well as generalized claims, and you can find them on the MySQL benchmarking page here: http://www.mysql.com/why-mysql/benchmarks/

“Big-Iron” capabilities Many people bemoaned (or denigrated) earlier versions of MySQL because it was missing this feature or that one which, while not necessarily used, were always on the ‘checkmark’ list of evaluators. Bemoan no longer – they’re here! MySQL 5.0 includes the following ‘enterprise-strength’ features: Stored Procedures Triggers ACID Transactions • • •


Views Information Schema (meta data) tables Distributed Transactions (across multiple databases) and other ‘big-iron’ features, such as the ability to create a single logical database from multiple physical servers. Naturally, the MySQL people are proud of the work they’ve put into their latest version, and accordingly the MySQL web site has buckets of details on each of these features. • • •

Business reasons There are multiple business reasons for using MySQL, and surprisingly, they don’t all revolve around money.

Cross-platform capabilities Remember when Microsoft used to tout the cross-plat capabilities of Fox when it ran on DOS, Windows, Unix, and the Mac? And then somewhere along the line, the marketing folks got a hold of the term “cross-platform” and subverted it to mean “multiple versions of Windows”: “Yes, VFP is cross-platform – it will run on Windows 3.1, Windows 95, AND Windows NT!” Reminds you of the “We have both kinds of music – country and western” joke from the Blues Brothers. You can develop and deploy your MySQL database on a wide variety of operating systems, including Windows, Linux, Macintosh OS X, Solaris, FreeBSD, HP-UX, IBM AIX, Novell Netware, and others. This means you’re not tied to the whims of a specific operating system manufacturer. Nor are you tied to a specific box of hardware for development or deployment – instead being able to follow the lead of Google, who relies on large farms of commodity boxes instead of expensive proprietary machines. Installed base MySQL’s growing popularity means its installed base continues to grow, which is beneficial in a number of ways. First, more users mean more scenarios are being tested, which means a stronger product. More users mean a larger community of support. And more users mean a more attractive market for third party tools and vendors, which makes the product more flexible and able to meet unusual needs. Because of MySQL’s inexpensive licensing, more and more companies are also including it as the database engine in their products – oftentimes unbeknownst to the end user! This further grows the pool of customers. Inexpensive MySQL’s immediate attraction to many is that it costs nothing to develop with – it can be downloaded and used for free. Deployment, while not always free, is considerably less than a similar deployment with Microsoft SQL Server or Oracle. That means if you want to kick the tires, try some things out, experiment, or even try a test deployment or three, you don’t have to shell out an arm and a leg for the complete system. Proprietary manufacturers often offer a ‘free’ version of their expensive software, but check to see if it’s truly identical to the licensed version, or if it’s crippled in some way that makes it an inappropriate choice for evaluation or testing. Compare the costs for downloading and installing a half dozen or dozen copies of MySQL for your development to the cost of development licenses for various MySQL competitors.

12


There are some scenarios where you’ll need to pay for MySQL licenses, typically involving deployment. But even then, the apples-to-apples comparison of license costs is far in favor of MySQL compared to the competition. When the huge cost differential is brought up, competitors ignore the real world and start talking about high-end features exclusive to their products, regardless of whether or not those features are needed.

Freedom to recompile from source MySQL’s open source license means you have access to the source code. Some folks look askance at this argument, saying they’ve never met anyone who has modified an open source package, but that’s not the point. If you are having problems with a proprietary software package, you are held hostage to their interest in helping you out. If you are having problems with an open source package, you are free to find a solution yourself – you aren’t breaking the law if you want to do so. While admittedly precious few actually go in and make changes to the source code themselves, it’s not uncommon to recompile the packages yourself in order to make them work in a special configuration. For example, suppose your environment is tied to the use of another software/hardware combination, say, via a proprietary application your business depends on, and that combination is somewhat dated or otherwise specialized. You can download the install (or even download the source and build the server application from scratch yourself) for a long list of older versions. By contrast, it may be nigh-on-to impossible to get the supported versions of a proprietary database to work on that combination of hardware and software. In fact, to attempt the same thing with proprietary software is strictly prohibited by their manufacturers. And the flexibility and freedom of open source is an advantage to your business.

Arguments against There are always going to be those who will argue against the VFP/MySQL combination. Their reasons range from the logical (“We’re an Oracle shop and can’t afford to retrain our entire staff just to save a few bucks on licenses.”) to the political (“My boss likes Microsoft so that’s what we use, regardless of how it works.”) to the insane (“The CEO’s ex-partner owns stock in XXX, so we’re under orders never to let their stuff in our shop.”) As arguments go, the choice of computing platforms can get pretty heated, but when the smoke clears, the arguments against boil down to a few time-worn claims. Let’s take a look at those and help you deal with them.

FoxPro is dead (or obsolete, or not supported, or...) It’s rare when a FoxPro detractor will not immediately point out Microsoft’s long time lukewarm (at best) attitude towards VFP. Nearly since the day Fox Software was bought by (or ‘merged’ with, depending on which press releases you read) Microsoft, the rumors that Microsoft was going to ‘kill’ VFP have floated around. That was in 1992. 15 years later, VFP has gone through six major releases and had close to a hundred related releases in terms of foreign language editions, service packs, and other updates. You would kinda have thought that if Microsoft really bought Fox with the intent of discontinuing it, they would have been able to get around to it by now. Witness Microsoft Bob and ActiveDocs for examples of Redmond technology that got unceremoniously dumped. It’s true that Microsoft hasn’t promoted VFP the way the VFP developer community has wanted. No argument there; I’ve been one of the loudest complainers. But there’s a big


difference between lack of promotion and lack of support. As of this writing, Microsoft has committed to supporting VFP 9.0 through 2015. Name one other Microsoft development product that has a written support plan that extends anywhere near that time span. In addition, should Microsoft, eight years hence, decide to not extend support past that time, that decision is not a signal that VFP apps will stop running the next day. Even now, in 2007, there are plenty of FoxBASE and FoxPro apps, a dozen or more years old, that continue to serve the needs they were designed for back when the Internet was merely a gleam in Al Gore’s eye and a cell phone was exclusively a business executive’s domain, not an extension of every teenager on the planet. We can be comfortable knowing that we can expect our VFP 9 applications to be running for a decade or more. After that? It’s quite possible that a rewrite will be in order – doncha think that your businesses might have changed significantly by then?

Visual FoxPro won’t have a native 64 bit version Microsoft has stated that Visual FoxPro will remain a stand-alone, Win32 based application, and will run on 64-bit Windows in 32-bit compatibility mode. This, detractors claim, is yet another reason not to use Visual FoxPro. First of all, Microsoft also stated there was no way they would release version 7 of Internet Explorer independent of the next version of Windows. Months later, they did a 180 on that statement. They also claimed they couldn’t separate Media Player from Windows. But lo and behold, that tune has changed as well. So while this is what they’re saying now, that could change. But let’s suppose they don’t. So what? What they are saying is that VFP apps will run fine on 64-bit Windows, albeit in a different mode. What’s the problem here? Will VFP programs that are running natively on 32bit processors run slower in a 64-bit emulator? Some folks may claim this will be the case, but until they can prove it, I remain skeptical. There are many, many FoxPro for DOS applications running in a DOS box on 32-bit Windows now – and they’re actually blindingly fast. And those systems have had a lifespan of 15 years or more. No news here, folks, move along.

ODBC is too old/slow Once in a while I hear the “ODBC is sooooo yesterday’s-news” argument trotted out. They say that it’s too old to be useful. Alternatively, they’ll say it’s too slow. Both are bunk. ODBC is an open standard for defining access to relational databases. It is not reliant on individual database functionality or a proprietary API. All ODBC drivers support the same set of commands and functions so code that works with one is sure to work with another. This is not the case with OLEDB, which is totally reliant on native APIs – for example, the commands for the WORD provider are different from those for SQL Server! ODBC is cross platform and not reliant on any one operating system technology. OLEDB is COM based and is therefore dependent on Windows – the only OS to implement COM. Furthermore, ODBC is an international standard not controlled by any one company – though of course, any company can decide whether or not to make ODBC drivers for its products available

14


As far as speed... For the types of applications Visual FoxPro and MySQL will typically be used for, ODBC speed is not going to be an issue. The MSDN article “Is ODBC the Answer?” states:

ODBC was not designed to completely replace native database APIs …. Native database APIs do a better job than ODBC of exposing the capabilities of a particular DBMS and often expose capabilities not exposed by ODBC.… So which applications are candidates for ODBC? The best candidates are applications that work with more than one relational DBMS. This includes virtually all generic and vertical applications and some custom apps. Only high-volume transaction processing systems should be worried about ODBC speed issues, and they are usually not built with VFP and MySQL. Finally, MySQL AB is constantly updating their driver, both to improve performance as well as deal with bugs that show up.

MySQL isn’t powerful enough Powerful enough for what? No, you might not want to move AT&T’s 26 terabytes or FedEx’s 9 terabytes over, but who’s asking you to? The large, large majority of applications will fit in fewer than 25GB, which is certainly well within MySQL’s capabilities. Go up to the MySQL forums and nose around; you’ll find plenty of folks building systems with over ten million rows, and more than a few with 100 million rows. In some cases, they are not comparing apples and apples. MySQL detractors will use a variety of tactics, including using old versions of MySQL or ignoring capabilities of the most recent version. “MySQL doesn’t have stored procedures” is still a common refrain from many competitors. “Oh, they do? Well, they haven’t had them for long!” is about all they can respond with. Hardly a valid technical argument. In many other cases, well, let’s be frank, do you really need all the advanced features? Much like word processors, the most recent versions of all the database products are feature packed so as to meet reviewer evaluation guides – but the typical user (or developer) only uses 5% of them. Same for database servers – Microsoft, Oracle, and IBM pack their servers full of features that only 1% of their users need, but are added to satisfy those few enterprise customers. The huge majority of users use only a fraction of the available features. Do you really need clustering or the ability to distribute transaction processing across multiple databases and machines? The typical MySQL application really doesn’t.

MySQL’s ‘SQL’ is not compliant or standard One reviewer related the situation where a group deliberated on which non-VFP database they were going to use when rewriting part of an application. The option of MySQL was brought to the table. At the time, MySQL’s chief detractor at the table, a developer with SQL certification from Microsoft, argued that one important reason not to choose MySQL was that MySQL’s ‘SQL’ is not as ANSI 92/99 compliant or as standard as Microsoft’s SQL. This developer claimed that if you opted to use MySQL from the outset and eventually decided you wanted or needed to up-size to a commercial alternative you would more than likely be rewriting your


queries, stored procedures, and triggers because the SQL syntax would be broken in a DBMS like Microsoft SQL. MySQL has the following to say regarding the issue of standards compliance, at http://dev.mysql.com/doc/refman/5.0/en/compatibility.html “One of our main goals with the product is to continue to work toward compliance with the SQL standard, but without sacrificing speed or reliability. We are not afraid to add extensions to SQL or support for non-SQL features if this greatly increases the usability of MySQL Server for a large segment of our user base.” Admittedly, there are features in MySQL that go beyond ANSI standards. These features and the functionality they provide would assuredly be broken if that code were run under Microsoft SQL, Oracle, or any other database. However, if this is a serious concern to you (how many folks move from one back-end to another without spending significant amounts of time reworking the application itself as well?), you can avail yourself of MySQL without permitting yourself to use features or syntax that fall outside the standard. The mysqld command used to start MySQL has --ansi and --sql-mode options that force the user to “Use standard (ANSI) SQL syntax instead of MySQL syntax”, as per the documentation here: http://dev.mysql.com/doc/refman/5.0/en/server-options.html Finally, you can still write code that is more easily ported to another back-end via appropriate documentation, says their documentation at: http://dev.mysql.com/doc/refman/5.0/en/extensions-to-ansi.html MySQL Server supports some extensions that you probably won’t find in other SQL DBMSs. Be warned that if you use them, your code won’t be portable to other SQL servers. In some cases, you can write code that includes MySQL extensions, but is still portable, by using comments of the following form: /*! MySQL-specific code */"

Finally, every version of SQL has their own proprietary, non-standard extensions. The three major DBMS vendors, IBM, Microsoft, and Oracle, all seem to offer features and syntax variations beyond standard and the idea of true portability seems largely fantasy to begin with. See Peter Gulutzan’s article here: http://www.dbazine.com/db2/db2-disarticles/gulutzan3 You’ll see that claiming MySQL is not as compliant as another database is a straw man – they’re faulting MySQL for something they themselves do just as well. It is obvious that maintaining portable MySQL syntax requires forethought and discipline, but this goal is certainly achievable. I would argue that the entire objection probably has relevance in a small minority of cases.

Conclusion/Summary The combination of Visual FoxPro and MySQL offers a powerful, easy-to-use, fast development tool set for building a large percentage of client-server applications needed

16


today, with extremely cost-effective licensing that means fewer dollars out of your, or your customer’s, pockets. Updates and corrections to this chapter can be found on Hentzenwerke’s Web site, www.hentzenwerke.com. Click “Catalog” and navigate to the page for this book.

Chapter 2: Development and Deployment Scenarios 17

Chapter 2 Development and Deployment Scenarios Given that MySQL is a true cross-platform database that runs on Windows, Linux, and Macintosh (as well as others), there are many combinations of environments in which VFP works with MySQL. Together with the segregated requirements of client-server components and the disparate needs of a VFP/MySQL developer during development, testing, and deployment, it can be confusing to understand what pieces are required and where they fit. This chapter discusses the various scenarios, what the components for each combination are, and how they fit together.

The configuration of simple Visual FoxPro applications is fairly easy to understand – an EXE, some run-time files, and a few DBFs. But client-server applications using VFP and MySQL are more complex. While I’ll go into detail about installation on Windows and on Linux in Chapters 3 and 4, it can be helpful to get a birds-eye view of how the various pieces fit together before going into that detail. Let’s take a look at the big picture first.

The big picture of how pieces fit together In brief, for your development configuration, you: 1. Install Visual FoxPro on a Windows box, 2. Install the MySQL ODBC driver on the same Windows box, 3. Install the MySQL engine either on the same Windows box or a separate box (running Windows, Linux, Macintosh, or another OS), and 4. Put the database on the same box as the MySQL engine. If the MySQL engine is on a separate box from VFP, all that is required is for the box to be reachable by the Windows box via a TCP/IP connection. Now that the pieces are in place, you connect to MySQL from VFP through the ODBC driver. Once connected, you can use remote views, SQL pass-through, or cursor adapters to bring selected data into your application, work with that data for a while, and then stuff it back into the database. After you’ve developed, tested, and staged your application (I know, this sounds like one of those “Step 3. Then a miracle occurs” jokes), you deploy the end result to production. In production, the MySQL engine will reside on the ultimate server box (again, running Windows, Linux, Mac, or another OS). The production database will also reside on the server box. Your VFP executable, VFP run-time files, and the MySQL ODBC driver will be installed on the client (end-user) Windows boxes.

The components There are three groups of components that we’re dealing with: Visual FoxPro, the ODBC driver to connect to the MySQL database, and MySQL.

18


Visual FoxPro As development environments go, the VFP architecture and installation requirements are rather straightforward. With the exception of a couple of runtime DLLs that go into the Program Files\Common Files\Microsoft Shared\VFP directory, the entire installation ends up in Program Files\Microsoft Visual FoxPro 9 directory by default. VFP installs off of the CD. The VFP IDE is shown in Figure 1.

Figure 1. The Visual FoxPro IDE with the Command Window, Data Session, and two debugging windows. The IDE provides access to the programming language via the menu and the Command Window, as well as all of the visual tools, such as the Form Designer and Report Writer, and the development tools, like the Object Browser and Debugger. Once installed, you can customize VFP’s behavior through the Tools | Options menu, but no specific configuration is required to use VFP with MySQL. There are three ways to use VFP to connect to MySQL (all using the ODBC driver). These are (1) remote views, (2) SQL Pass-through, and (3) CursorAdaptors. You don’t have to do anything additional, or install any other software, in order to use any of these mechanisms.


ODBC driver ODBC (Open Database Connectivity) is a standard database access method developed over a decade ago and in widespread use even today. The goal of ODBC was to provide a bridge between data and applications, using an intermediary called an ODBC driver. It is the link between Visual FoxPro and MySQL, regardless of whether they are on the same machine or on boxes separated by half the planet. You can set up connections to different databases and switch between the databases simply by changing the connection your application uses. The MySQL ODBC driver is a standard ODBC driver, consisting of a DLL and a LIB file that both end up in the Windows\System32 directory. Once installed, you’ll find it in the Drivers tab of the ODBC Data Source Administrator as shown in Figure 2. (To find it, select Start | Settings | Control Panel | Administrator tools | Data Sources (ODBC) in Windows 2000, or Start | Settings | Control Panel | Performance and Maintenance | Administrator Tools | Data Sources (ODBC) in Windows XP.) I’ll cover installation and configuration of the ODBC driver in Chapter 6.

Figure 2 . The ODBC Drivers page with the MySQL ODBC driver highlighted.

MySQL Let’s not beat around the bushes – there are a lot of parts to MySQL. To understand them, it’s helpful to know how MySQL itself works, in a general way. MySQL is a software application that runs as a service (known as a ‘daemon’ in Linux parlance) on a computer. This means that once it’s installed and started, it waits in the background for a request and returns the result of attempting to carry out the requested action. This could be a value, a result set, or an error message. Similar to the way a Web server takes requests from a browser and delivers Web pages back to the browser, the MySQL server takes requests from a user, queries or modifies the database as a response to those requests, and finally returns those results to the user. The server is just a program (either ‘mysql.exe’ on Windows or ‘mysqld’ on Linux) that can be started up automatically when the machine is booted, or manually by issuing a

20


command in a command window. During startup, it looks for information about how to configure itself from a configuration file. It creates a socket file (provides an anchor for a connection) and a PID file (a placeholder that identifies which instance of MySQL is connected to the databases, as a machine can run multiple instances of MySQL). Once running, it waits in the background for requests from users and performs them on databases in a predefined location. The software that supports these functions can be grouped into five general categories: 1. Database engine 2. Database files 3. Client tools 4. GUI tools 5. Scripts First is the MySQL database engine itself. On Windows, it’s mysql.exe, located in the \bin directory under wherever you install MySQL – either \Program Files on Windows or on Linux, it’s /usr/var/mysql/mysqld. (Executable files on Linux don’t necessarily have extensions.) Next are the database files. You can determine where they are located during installation. Unlike VFP, MySQL has several different types of storage engines, including the MySQL ISAM type and the InnoDB type. The number and types of files vary according to what type of storage engine you choose. Third are the client tools. These allow command line access to the database engine, somewhat similar to the way you can use commands to access and manipulate Fox data. Fourth are the GUI tools. These, similar to the client tools, allow access to the database engine, but via graphical interfaces. These tools include the MySQL Administrator, the Query Browser, and the Data Converter. Their locations can be chosen during installation, although I typically put them in directories off of the mysql installation, such as c:\mysql\bin\admin and c:\mysql\bin\qb on Windows, and similar paths on Linux. Finally, we have the scripts (Windows users – think ‘batch files’). These scripts are installed along with the MySQL database engine installation and perform tasks like creating privilege tables, setting permissions, converting tables to ISAM format, securing an installation, and performing a hot backup of a production MySQL database.

The scenarios I recently spent some time with my 8th grader going over combination problems in her math class. You know the type – “Jack has four pairs of pants and six shirts – how many days can he wear a different combination?” The same type of math generates a very large number when applied to the various permutations of VFP and MySQL on various operating systems and machines. Instead of trying to go through each of them, I’ll approach the problem a different way. I’ll cover development scenarios first, including testing requirements, and then revisit them in terms of what’s different for production.

Development VFP First, you’re a software developer. You have a machine that’s running Visual FoxPro. While you can technically run VFP on Linux (via WINE or Crossover Office), that’s not a common


option, so I’ll assume this box is running Windows – either Windows 2000, Windows XP, or Windows 2003, as Windows Vista was just barely released during this writing. Second, you have the MySQL ODBC driver installed on that Windows machine. And now it gets interesting – you have multiple choices for where MySQL is located. There are several good reasons to put the engine itself on the same machine as the development environment, and other good reasons to put the engine on a different box. I’ll cover the pros and cons for each in the appropriate section.

MySQL on the local Windows machine Choice number one is to install MySQL on the Windows box as well. I prefer to run the engine on the local box for the simple reason of expediency. Response is faster – while network connections are fast, being on the same box is virtually immediate. As I heard a long, long time ago, the key to successful iterative development is to keep the iteration turnover shorter than your attention span. And as one gets older, one’s attention span gets shorter, so every trick in the book to speed up that iteration time helps. Second, my development becomes portable. I personally develop on a notebook, but even if you use a desktop or other semi-luggable box, being able to pick up and move your entire development environment can come in handy, either because you must (power failure, water pipe explosion overhead, office closure, whatever) or because you want to (going to the cottage for the weekend, heading over to the library or coffee shop to meet an associate, whatever). If your development back end is on another box, either you have to rely on a network connection wherever you’re going, or deal with tearing down, moving, and setting up two boxes. It may not be a big deal to you, but to me, it’s time that I could spend doing other things. Second, relying on a second box adds at least two more points of failure – the network connection itself as well as the other box. When I’m developing, failures cost money. Putting more potential points of failure in the development process is unnecessarily risky. Third, incorporating a network connection adds additional complexity to the environment. It may not be a LOT of complexity, but it’s still there. And having to deal with the troubles that additional complexity can bring during heads-down development is an unnecessary distraction. Note that installing MySQL on your local box does not absolve you of having to set it up on a separate box for testing! If you choose the single machine installation route, after installing VFP and the MySQL ODBC driver on your local box, you’ll also install the MySQL server and client packages, and then whichever of the MySQL GUI tools that you want. I’d suggest both the Administrator and the Query Browser, as they don’t take up a lot of space and are pretty handy, particularly for folks who are used to GUI tools. Scripts will get installed along with the server and client packages automatically. During the installation of the server, you’ll be asked where you want to put the MySQL database files – in other words, your application’s data. The default on Windows is under the MySQL software tree, which to my way of thinking isn’t a great idea – your data never belongs on drive C, nor in the software’s installation tree. I keep a separate drive on the local box for my data (or at least a separate partition, if the box only supports one physical drive), and if you follow the same practice, you’ll want to create a location on your data drive for your MySQL databases, and point your MySQL data in that yonder direction.

22


Some will argue that doing all of your development on a local machine can give you a false sense of security in terms of how fast your application is – you spend months developing locally, and then you roll it out for a customer to see and suddenly find out where the bottlenecks are. It can be embarrassing. My answer to this scenario is that I always have testing done with a remote server. A testing environment is much more like production will be than what my development environment will be. And the test folks are plenty willing to provide feedback on how the system performs.

MySQL on a remote machine The other choice is to put MySQL on a second (“remote”) box. There are a number of good reasons to have your MySQL installation on a separate box – and it doesn’t really matter what the OS is running on the second box. First of all, you’re going to have to connect to a second box from a client at some point, because your application will need to do so in production. No one wants to be the fellow who shrugs in front of the customer, saying, “Well, it worked on my machine...” Second, you don’t want to try to do your first-ever remote connection the night before “The Big Install.” By setting up a separate box early in the game, you’ll have the luxury of debugging problems without the spectre of time running out looming over your head. Just as you don’t want to be the “Worked on My Machine” guy, you don’t want to be the guy who is up till 4:30 am the morning of the “Big Delivery To The Customer,” trying to puzzle out why your system won’t connect via panicked emails to a mailing list where the only folks on-line are a) folks in a time-zone 12 hours away, with the attendant possible language differences, or b) similarly panicked folks like you, who won’t be of much help. Third, once your remote box is set up, you can emulate your customer’s environment more closely, and that’s always a good thing. You’ll be able to test more accurately, and it makes troubleshooting easier and more reliable. Also, you can verify that your network’s speed can handle the traffic your application will add. Properly done, you can even do load testing to make sure your application can handle as many users as will be necessary. Additionally, since you’re going to have a second box, you might as well set it up and then put it in the server room, the basement, the attic – wherever it’s a little bit inconvenient to have to physically get in front of it. Thus, you’ll be forced to learn how to administer the box remotely as well. While your current gig might not require you to develop this level of knowledge, who’s to say your next one won’t? And it’s not unheard of to have a customer swear up and down that they will take care of the administration once it’s installed, but the next time you visit, their DBA is “unavailable” (that’s French for ‘too busy’, ‘sick’, or ‘fired’) and “could you just take a look at one thing?” You become the hero if you have the capability to get into their system as a remote administrator. Finally, there’s also the advantage of security. A client’s data may contain sensitive information: names, SSN’s, birthdates, medical information, and operational information. If the data is on your laptop, it’s subject to theft with that oh-so-portable laptop of yours. If it’s locked up tight on a server behind a firewall with secure access, strong passwords, and good security processes, it’s less likely to get pinched when you step away just for a second to refill that latte grande. This entire discussion so far has been based on the assumption that you’re the only developer working on the system. If you have more than one developer working on the


application that needs the database, the choice becomes moot. You’ll need to put the database on a server that everyone can access. And since you’re likely not going to want others connecting to your own development box to access your local database server, the logical choice for the database is a second box. Expanding on this last point, you can also easily poke a hole in your firewall (say, limited to one IP address) to allow your customer to access the remote database, say, for demonstration or testing purposes. You probably wouldn’t want to do that with your local development box. Finally, it is entirely plausible that you will be involved in using a MySQL database hosted on an ISP’s site. In that scenario, you will definitely need to be able to do administration remotely. The entire preceding argument has sort of danced around the question of whether you use the remote box for your day-to-day development. Some people swear by the use of a second box for their day-to-day work, and there’s a valid argument here – the more hours you spend working with a remote box, the more experience you’ll get, and in the long run, that experience will be good for you. If you choose that route, the $64,000 question becomes “OK, what goes where with a remote box?” This explanation actually works with all flavors of a remote box – Windows, Linux, Mac, etc. You’ll install the MySQL server (the ‘engine’) and client tools on the remote box. Doing so will also create the scripts as well as the database on the remote box. The GUI tools are designed to be used when you’re physically in front of the machine, so you don’t need to install them on the remote machine at all. Sure, you could, if you wanted to use GUI tools to administer the remote box during those times when you’re not connecting to it remotely, but, rather, sitting right in front of it. But after you get the remote box running (using the MySQL monitor, which is part of the client package), you’re likely to put it in a closet, take the head off (that means remove the monitor, mouse, and keyboard – a common scenario for Linux servers), and not touch it again until the cooling fan or hard drive smokes. Then all of your future administration will be handled from another machine. “But I LIKE the GUI tools,” you whine. Never fear. So do I. You can still use GUI tools on one box (say, your development box) to access the database server on the remote machine. Some would argue that having the GUI tools installed on the server used for development can be a good thing, in order to troubleshoot locally with the tools you’re familiar with. Maybe the server isn’t responding to remote access. Or features and functionality seem to be missing. Having to figure it out from a remote command line can be a nuisance. I personally don’t go this route, but it’s more of a personal preference than an absolute.

Production Production is a different beast than development, in the same way that the country club where your daughter is getting married is different than the family room that opens up onto the deck in your backyard. Well, at least for me. The production machine is clean and orderly, with no ‘half-finished’ projects lying around, while the development machine is always in a state of mild disrepair. Still, the same separation of function basically applies – you’ll have client machines running the Visual FoxPro application, and a separate server machine running MySQL and holding the database.

24


Client machines On your development machine, you’ll build an executable (. EXE) file from Visual FoxPro that contains all of the functionality the user wants in your system. You’ll install that EXE, along with Visual FoxPro runtime files and the MySQL ODBC driver, on each client machine. For all but perhaps the simplest of applications, you’ll use an installation tool like InstallShield or InnoSetup to handle all of the plumbing – registering DLLs, configuring components, and installing all the pieces that tend to get left behind if done manually. Some folks also put together an automatic updating utility (written in VFP, of course) that allows the developer to centrally manage the deployment of updates. You won’t install the Visual FoxPro development environment, nor will you install any of the administration or GUI tools for MySQL, unless you’re expecting to have to use one of your client workstations for that purpose. This means you don’t need to buy any sort of license or pay royalties for the clients – which can mean considerable savings over software like Microsoft SQL Server that requires client access licenses for each user. Server machine The server machine will hold the MySQL server application and the database files. Thus, you’ll perform the server and client software installations on the server machine, which in turn will take care of the database creation for you. You’ll need to configure the server for your particular needs and deploy your applications database yourself, unless you’ve put together an automated installation routine that does it for you. As with client machines, the details to create such an automated routine are beyond the scope of this book.

Conclusion/Summary All things considered, if having your MySQL on a remote machine similar to your production server is not a major inconvenience, and doesn’t pose an unreasonable barrier to speedy dayto-day development, I recommend doing so for all the practice you will have with the remote environment and the realistic response. It is important to have experience with a remote server so you don’t get lulled into a false perception of performance for the application. At the same time, you don’t want to unnecessarily handicap yourself during development when performance can be tested at the appropriate time. Updates and corrections to this chapter can be found on Hentzenwerke’s Web site, www.hentzenwerke.com. Click “Catalog” and navigate to the page for this book.

Chapter 3: Installing MySQL on Windows 25

Chapter 3 Installing MySQL on Windows Installation is like finding the emergency brake in your car. Once you know where it is, it seems rather trivial. However, if you can’t find it, you’re not going very far. In this chapter, I’ll show you how to install MySQL on a Windows box, starting with downloading the right files, and finishing up with some troubleshooting steps. In subsequent chapters, I’ll describe the connection process between MySQL and VFP.

This chapter covers the installation of the MySQL server on a single computer running Windows. There are several ways to install MySQL. You can either install it via a standard Windows installer, much like you install most other Windows programs, or you can build the executable files yourself from source. This chapter describes how to use the installer, and then configure MySQL once installed. Note that throughout this chapter, I’ve included screen shots of the MySQL website. It is highly likely that the site will see modifications that render some, if not all, of these images out of date. However, I’ve found over the years that the essential items you’re looking for keep the same names – they’re just in somewhat different locations.

Software and environment This chapter was written using MySQL 5.0.18 and Windows 2000 with the latest service packs installed, but I’ve used these same steps, with minor differences (such as version numbers of the downloaded MySQL files) to install other combinations, all the way back to MySQL 3.23. When appropriate, I will mention the differences between Windows 2000 and Windows XP. In the examples used in this chapter, the Windows box is named ‘example’, with an IP address of 192.168.1.7 and a day-to-day user named ‘whil’. The user account in MySQL will be named ‘bob’. (This Windows box is also running Visual FoxPro, but in Chapter 5, “User Configuration,” I’ll address the concepts for communicating with a remote Windows box.)

The big picture – what installation does If you haven’t already, read Chapter 2 on how MySQL works to make sure you understand the context. MySQL on Windows consists, in a general sense, of two sets of files – the server and the client. The server installation process installs the MySQL server – the actual software application. The Windows version of the MySQL server application itself is mysqld.exe, located in a directory of your choosing, such as c:\Program Files\MySQL Server 5.0\MySQL\bin\mysqld.exe. The client installation puts a number of client programs and scripts (think ‘batch files’) in directories on the same level as the ‘\bin’ directory under ‘mysql’, wherever you decided to put it. Then the installation process sets up the server as a service, enables the service to be started automatically upon boot-up, creates user accounts and data files, and turns over the keys to you.

26


The essential installation process Installation of MySQL involves five steps: (1) Read the documentation on the MySQL site, (2) Download the appropriate file(s), (3) Run the installer, (4) Test the installation, and (5) Configure users and set passwords. Because the user configuration and password setting process is the same for both Windows and Linux, I’ve given it its own chapter following Chapter 4, “Installing MySQL on Linux.”

Read the documentation Since you’re reading this document, you’re not too adverse to reading the fine manual that comes with the product, and in this specific case, I urge you to take advantage of it. The online documentation is very good, and you shouldn’t take this document as a substitute for it. However, the documentation has to cover all possible scenarios, such as installing various types of files (RPMs and tar files) on Windows, Linux, and other operating systems, so it can be confusing when having to skip parts that are not related to your particular situation. Thus, I winnowed out the parts not relevant to this particular type of installation, but point you to specific sections in the documentation when the time comes. So look at this document as a supplement to the online documentation, not a replacement. The MySQL folks continually change the interface of the www.mysql.com website, and so it’s impossible to provide specific steps that will work for finding the documentation regardless of what year you’re reading this. Either go to http://www.mysql.com and look for “Developers” or “Community” links, or go to http://dev.mysql.com (hopefully this URL won’t change for a while) and look for a link for Documentation. I found the documentation to the right of the “HTML, viewable online” links (separate versions for the different releases of MySQL) to be easy to navigate and very useful, as the user comments that are often added at the end of many sections provide additional tips and a variety of perspectives of real life usage. Consider adding a direct link to the online documentation as a bookmark in your favorite browser.

Download files Before you actually download the MySQL installation files, you should decide where to put them.

Decide on a target location Personally, I put all of the files I download into a directory called “zips” on my file server. Once I find that I’m actually using a program or tool, I’ll archive the installation file(s) to the server for backup. It can be awfully inconvenient to want to reinstall something and find out that you can’t get to the vendor’s website. Also, if you’re using Firefox and have the downloading window turned off, you can temporarily turn it back on via Tools | Downloads, as shown in Figure 1.


Figure 1. Opening the Downloads window in Firefox. Doing so will open the download window, as shown in Figure 2.

Figure 2 . A sample Downloads window in Firefox.

Grab the files Once you decide on your own storage location, it’s time to find the files. Like the documentation, the links tend to move around over time. There are two general versions, the Enterprise Server and the Community Server. Further complicating the matter is that at any given time, there are multiple versions of both – older versions (3.x, 4.x), the current stable

28


version (often referred to as the Generally Available Release), and one or more alpha and beta releases. For the purposes of this book, you’re looking for the Generally Available (GA) Community version. Navigate to http://dev.mysql.com, click on Downloads, and look for a reference to the Community Server. Scroll down to the link for Windows, click on it, and you’ll see a list of Windows-related files as shown in Figure 3.

Figure 3 . Navigating to the Windows downloads for the MySQL engine. You’ll see links to three possible choices: Windows Essentials (about 17 MB) Windows (x86) (about 33 MB) Without installer (unzip in C:\) (about 35 MB) Select the middle link – Windows (x86) – that contains the complete package of files. If you’re short on disk space, the Essentials download contains the minimum set of files needed to install MySQL on Windows, including the Configuration Wizard. Before you actually grab the file itself, I suggest you cut and paste the MD5 text string from this page and pop it into an empty text file editor window; you’ll use this in a few moments. Click on the ‘Pick a mirror’ link on the far right. The “Select a Mirror” page displays, as shown in Figure 4. • • •


Figure 4 . Navigating to the mirrors for the MySQL files. If you look carefully (the print is small), you’ll see another link that says You are downloading mysql-5.0.nn-win32.zip

where ‘nn’ is the minor version of the file you’re downloading. If you click on the ‘No thanks’ link, you’ll be scrolled down to a list of mirrors, as shown in Figure 5.

Figure 5 . Selecting a mirror for the MySQL files.

30


Click on an HTTP or FTP link as you desire, and you’ll download a file that is named something like mysql-5.0.15-win32 - The actual server install for windows (38 mb)

This is the complete package with lots of extras AND a nice installer.

Verify the bits It’s a good idea to verify the integrity and authenticity of packages before installing them, as the bits can get scrambled while they’re being downloaded. See the on-line documentation’s section 2.1.4 “Verifying Package Integrity Using MD5 Checksums” for details. The package “winMd5Sum” is a graphical MD5 checking tool that can be obtained from http://www.nullriver.com/index/products/winmd5sum. Remember the MD5 text string you cut and pasted into a text editor window? Any MD5 checking tool you use will need this string – and now you have it handy, instead of needing to go back and find it again!

Uninstalling an old MySQL installation If you already have MySQL installed, you may want to uninstall it before continuing. Note that you don’t have to uninstall a previous version, mind you, because you can have multiple versions of MySQL running on the same box, just as you can have multiple versions of Visual FoxPro running on the same box. If you don’t have a previous version of MySQL installed, or if you want to keep it on your machine in addition to this new version, you can skip ahead to the “Run the Installer” section.

Stop the service If MySQL is installed as a service, you’ll need to stop the service. (You can tell if this is the case on Windows 2000 by opening Control Panel, opening the Administrative Tools folder, and selecting the Services applet. On Windows XP, the Services applet is found through Control Panel, Performance and Maintenance, Administrative Tools, Services. If MySQL is installed as a service, you will see it in the alphabetical list of services.) You can stop the service in a DOS box, like so: c:> NET STOP MYSQL

or you can use the Services applet just mentioned – click on the MySQL service, and either click the Stop Service button in the toolbar at the top of the Services dialog, or rightclick the MySQL service and select the “Stop” item from the context menu. If you are not running the MySQL server as a service (in other words, if you manually started the server and are running it like you would any other regular program), use a command like the following to stop the server: C:\> “C:\\bin\mysqladmin” -u root shutdown

where will be where the MySQL executable is located. This is the equivalent to doing “File | Quit” in VFP, for example.


Remove the service Next, you need to remove the service completely. You can do so via DOS: C:\> “C:\\bin\mysqld” --remove

or via the Add/Remove Programs applet in Control Panel.

Install the MySQL files with the installer Unzip the file you downloaded – the contents are comprised of a single file – SETUP.EXE. Run SETUP.EXE (it doesn’t matter where from), and you’ll see the initial Installer dialog, as shown in Figure 6.

Figure 6 . Starting the MySQL Installer.

The Setup Wizard Figure 7 shows the standard Welcome screen,

Figure 7 . The standard Welcome screen.

32


which is then followed by the Setup Type screen, displayed in Figure 8.

Figure 8 . The Setup Type dialog. If you select Custom, you can see what options are available as well as control where you want to install everything. Thus, I suggest you select Custom, as shown in Figure 9.

Figure 9 . The Setup Type dialog with the Custom choice selected.


Figure 10 shows the options you can choose to install (or ignore). If this is your first installation, I suggest you install everything if you have the space. In Figure 10, I deselected a couple options that I typically don’t bother with.

Figure 10 . Looking through the available options for a custom installation. An important piece in Figure 10 is easy to miss, so I make a big deal out of it now. It is where you choose to install MySQL. Click the Change... button and navigate to the directory where you want to put the MySQL executable and related files. I have, in the past, stayed away from the “Program Files” location inasmuch as previous versions of MySQL have had difficulties parsing the space in the folder name. Figure 11 shows an alternative “c:\mysql” installation directory.

Figure 11. Changing the Installation Directory during Custom Setup.

34


Clicking “Next” then shows the dialog that confirms the choices you just made, as shown in Figure 12.

Figure 12 . The selected settings confirmation dialog. Clicking Install will then display the traditional Installing d ialog with the thermometer bar scrolling across as informational messages flash by, as shown in Figure 13.

Figure 13 . The Installation dialog displays the standard thermometer bar and progress messages. Finally, you are greeted by the Sign Up dialog, as shown in Figure 14.


Figure 14 . Using the Sign-Up dialog is optional. Should you create a new MySQL.com account? I’ve found the various support options to be helpful, so I would recommend it. (You can always create an account later if you want.) If you choose to create a new account, you’ll be greeted with the new account page as shown in Figures 15 through 18.

Figure 15 . Entering login information during account signup.

36


Figure 16 . Entering demographic information during account signup.

Figure 17 . Entering subscription information during account signup. You can select the first checkbox if you want the folks from MySQL to contact you about licensing, and the third if you need to know about what’s happening at MySQL the moment it happens. Myself, a monthly newsletter and the MySQL forums are enough.


Figure 18 . Verifying information during account signup. Finally, you’ll see the Wizard completed dialog as shown in Figure 19.

Figure 19 . Choosing to configure the server during installation. If you check the “Configure the MySQL Server now” check box, you’ll launch the Instance Configuration Wizard as shown in Figure 20.

38


Instance Configuration Wizard While you can run the Instance Configuration Wizard later, I suggest you go ahead and do it now unless you have someplace else terribly important to be right now.

Figure 20 . Starting the Instance Configuration Wizard. Just like the Installation Type earlier, I suggest you select the Detailed Configuration, so you can see the various options available to you, as shown in Figure 21.


Figure 21. Choosing which type of configuration to set up. Since you’re installing on your developer machine, select that option, as shown in Figure 22.

Figure 22 . Choosing which type of server to configure. As you can see, the configuration wizard allows you to choose various types of installations, and will select the appropriate parameters to tune MySQL for the type of use that

40


the server is going to get. Choices in Figure 22 drive the memory allocation for MySQL while the database usage choice, shown in Figure 23, determines the initial choice of engine. Don’t fret too much about making the ‘wrong’ choice now. You can change these selections later if you like, described in Chapter 5, “Configuration.” MySQL has several types of database storage engines. The two most popular are MyISAM and InnoDB. InnoDB is a high performance alternative to the native MyISAM engine, and is installed as the default on Windows installations. I’ll talk more about the differences in Chapter 9, Under the Hood, as well as mention nuances in other places as they come up. Since you’re just getting started with MySQL, I would suggest you select the Multifunctional Database option button, as shown in Figure 23.

Figure 23 . Choosing the type of database engine to start out with. If you chose Multifunctional or Transactional Only, you’ll be prompted for information about where to put the InnoDB files, as shown in Figure 24.


Figure 24 . Choosing the location for the InnoDB database files. As mentioned earlier, I would suggest you change the default to a different drive to store your data files in a location other than buried in the bowels of your C drive. Figure 25 shows the selection of a custom directory named “mysqldata” that is under a generic “wsdb” directory used for all database files.

Figure 25 . Choosing a data directory for InnoDB files. Figure 26 then shows the InnoDB settings dialog with the new data directory selected.

42


Figure 26 . Confirmation of the directory for InnoDB files. Next you’ll be asked to choose the type of application this installation of MySQL will be supporting in order to determine how many concurrent connections will be set up by default. Because this is your first MySQL install, you’ll probably want to leave the default Decision Support selection chosen, as shown in Figure 27. If you have a bare bones machine that you’re using just for testing, you might want to select Manual Setting and notch the concurrent connections down to 10 or so, to save on machine resources. If you aren’t sure if you should change it, or you zipped by this too quickly in your first pass, or you later find MySQL bogging down on that clunker you hauled out of the closet for experimentation, never fear. You can also change this value in the MySQL configuration file, which will be covered in detail in Chapter 7, “Configuring MySQL.”


Figure 27 . Choosing the number of connections for the server. The next dialog, shown in Figure 28, allows you to determine how to connect to the database server. The defaults are to enable TCP/IP networking and specify port 3306 for the connection. (Why 3306? 3306 is the standard MySQL port, just as 80 is for HTTP and 443 HTTPS.) Leave this as is for the time being. You would only want to change this if your network requirements already used 3306 for something else (typically not a good idea) or if your network administrator specifically wanted to use a non-standard port for MySQL. (By the way, if you have a firewall running, be sure to open port 3306. Details can be found in the “Potential problems” section next.) Strict mode defines what SQL syntax is supported by the database server; selecting ‘Enable Strict Mode’ makes MySQL more compatible with other database server SQL syntax. Unless you have a specific reason to not want Strict Mode, keep this checkbox selected.

44


Figure 28 . Selecting the type of networking. Figure 29 shows the selection of the character set; like the previous dialog, unless you have a specific reason to make a change, you should keep the default selection of “Standard Character Set.” If your data set contains characters from a single non-English language, you can manually pick which character set you want MySQL to work with, while if you work with data that has characters from a wide variety of languages, you should select the Multilingualism option button.

Figure 29 . Selecting a character set.


Figure 30 allows you to configure the server instance you are installing. There are a lot of goodies in this dialog.

Figure 30 . Selecting the type of startup. The first option you have is to determine how you’re going to have MySQL running. If you choose to run it as a service, you’ll need to select the checkbox. Unless you have a specific reason why you don’t want to (for example, if you’re only going to use MySQL rarely, and are so short on memory that you can’t spare even the minimal amount of memory it uses), running as a service is the recommended method. You’ll probably want to configure your machine to launch the server automatically, too. An inordinate number of questions on the MySQL support forums ask questions that are answered with “Are you sure your server is running?” As I’ve mentioned, you can have multiple versions of MySQL installed on a single box. If you do, you’ll want to name each one uniquely. The Service Name combo allows you to determine what you’re going to call this instance. If you only have one instance of MySQL running on the box, you can get away with simply “MySQL”. However, if you already have versions on the machine, or if you think you’ll install additional versions later, you might want to pick a more specific description. The combo box opens to display choices that vary by version number, such as MySQL, MySQL 4, and MySQL 5. If you plan on running the server manually, by executing a command in the DOS box, it can be handy to have the \bin directory in the MySQL installation directory included in the Windows path (so you don’t have to explicitly include the entire path when you call the server.) Select the bottom check box if this sounds like something you want to take advantage of. Once the service is configured the way you want it, it’s time to move on to security, as shown in Figure 31.

46


Figure 31. Selecting initial security options. First, keep the “Modify Security Settings” checkbox selected, and enter a password for the MySQL root user. This password is what you’ll use when logging into the MySQL server with the username ‘root’ (as opposed to the username you use when logging into your Windows machine.) Ummm, need I mention that you should write this password down? If you don’t, and forget it, you’ll need to reinstall MySQL from scratch again, which can be a.... nuisance. Do not check the “Enable root access from remote machines” checkbox unless you explicitly have a very good reason for doing so. Since you enabled TCP/IP connectivity a few dialogs back, anyone who can access your machine via TCP/IP can attempt to crack into your MySQL server as root, and potentially take over the database. It’s a good habit not to even allow that hole to be open. Lastly, keep the “Create An Anonymous Account” checkbox unchecked as well. If you have to allow anonymous access to your server, it won’t be while you’re learning to use it, and you can turn this ability on at a later time. Finally, it’s time to write the configuration settings. The Ready to Execute dialog in Figure 32 will stay on the screen and show the progress being made after you click the Execute button.


Figure 32 . The Ready to Execute dialog tracks the configuration progress. The circles on the screen are not option buttons – they’re a somewhat unusual way of showing progress being made – the circle to the left of the prompt is checked off as each step is executed. If everything goes well, you’ll be greeted with a confirmation screen as shown in Figure 38. However, things don’t always go so smoothly. Let’s discuss a couple of potential problems.

Potential problems There are several types of problems that might occur during configuration, including firewall conflicts, anti-virus conflicts, and collisions with prior installations.

Firewall troubles First, if you have a firewall, you could get blocked with a MySQL error as shown in Figure 33.

48


Figure 33 . A Connection Error dialog will display if the server can’t connect. In some cases, it just takes MySQL a moment to catch up with itself, and simply pressing “Retry” will work. If you’re running ZoneAlarm, for example, you might see the alert dialog shown in Figure 34 right after the Connection Error from Figure 33.

Figure 34 . A typical ZoneAlarm error when a port is being blocked.


You can see that the MySQL daemon (mysqld-nt.exe) is trying to allow connections over port 3306. Select the “Remember this answer” checkbox and click “Yes,” and your firewall should be ready to allow MySQL to run.

Anti-virus conflicts Another problem that folks typically run into at this stage is anti-virus software being too aggressive (go figure!). If you aren’t running a personal firewall and still get the above error dialog, check if you have Norton Antivirus® installed. One of my reviewers reports finding that with Norton’s Internet Worm Protection enabled, you’ll get the same error as in Figure 33 above. To turn off the Internet Worm Protection, right click on the Norton Protection Center icon in the system tray and choose “Open Norton Protection Center.” This will open the dialog in Figure 35.

Figure 35 . The Norton Protection Center can cause problems on some installations. Choose the Security Basics link, which brings forward the dialog shown in Figure 36.

50


Figure 36 . Norton Security Basics dialog. Select Internet Worm Protection and turn it off. The dialog will give you the option to turn it off for a specified period of time. Choose “Until system restart.” If you’re running Windows XP, you may not be aware that the built-in firewall defaults to blocking port 3306, but does so silently, which can be frustrating. Kind of like a teenager giving you the silent treatment for some imagined slight. You’ll just get an error similar to the one shown back in Figure 33.

Collisions with a prior installation Next on our list of potential problems are a pair that are related. The reason I bring these up now instead of earlier is that they typically happen if you’ve already installed and uninstalled MySQL. After clicking the Execute button in Figure 32, you may end up with the “Could not start the service MySQL” dialog shown in Figure 37.


Figure 37 . The “Could not start service” error dialog. It’s particularly frustrating because of the helpful addendum, “Error: 0.” What does that mean? If you run into this, look in the mysql\data directory for the following three files: ib_logfile0 ib_logfile1 ibdata1

Deleting all three will allow the Windows configuration wizard to recreate them (evidently the wizard won’t automatically overwrite them), and the server should start properly. A related problem is the “Cannot create Windows service for MySQL. Error: 0” message. Note that it’s almost identical to the previous error – the difference is ‘create’ versus ‘start’. This error shows up when you reinstall MySQL without first stopping and uninstalling the existing MySQL service. When the Windows configuration wizard tries to install the service again, it finds an existing service with the same name, and pouts. See the section on uninstalling an old MySQL installation earlier in this chapter for details. If you have trouble removing the old service, you may find the following useful. A sharp fellow named Mike Hillyer created a service removal tool (named, interestingly enough, Mike’s Service Remover). You can find it at http://www.openwin.org/mike/index.php/faq/mysql/error-cannot-create-windowsservice-for-mysql-error-0/ Figure 38 shows what a completed, successful configuration dialog looks like.

52


Figure 38 . A completed configuration dialog.

Installation results Once the configuration has finished, now what? More directly, what’s happened?

MySQL files have been installed Let’s take a look at what has been installed and where. Figure 39 shows the directory structure of a typical MySQL installation in the directory C:\Program Files\MySQL.

Figure 39 . The directory structure created with a MySQL installation.


You’ll see the directory structure under the \MySQL Server 5.0\MySQL folder that includes ‘bin’, ‘data’, and ‘scripts’, among others. (Other directories, such as MySQL Administrator 1.1, are created on the same level as \MySQL Server 5.0 later, upon installation of other tools.) You’ll also note there are a number of ‘.ini’ files, including ‘my.ini’. Looking into the data directory, you’ll see the creation of some data files. \data\ \data\mysql\ \data\mysql\columns_priv.frm \data\mysql\columns_priv.myd \data\mysql\columns_priv.myi \data\mysql\ (about 50 more) \data\test\ \data\ib_logfile0 \data\ib_logfile1

We’ll examine these in detail in Chapter 9, “Under the Hood”. If you wandered over to the E drive, you’d see that, assuming you selected Multifunctional or Transactional Only as described back in Figure 24, there is also an InnoDB database started: e:\wsdb\mysqldata\ibdata1 (10,485,760 bytes)

The .ini files are discussed in Chapter 7, “Configuring MySQL.”

MySQL service created and started If you open up the Services applet (on Windows 2000 open Control Panel, then the Administrative Tools folder, and select the Services applet; on W indows XP, the Services applet is found through Control Panel, Performance and Maintenance, Administrative Tools, Services), you’ll see the MySQL service has been installed and started, as shown in Figure 40.

Figure 40 . MySQL installed as a service. You can see the name of the service, its status (Started, Paused, or empty – which means Stopped), what the startup type is – Automatic or Manual – and the user the service logs on as. The service should be set up as Automatic and already Started. You can right-click on the service name in the right-hand pane and then select the Properties item in the context menu to bring forward the dialog shown in Figure 41.

54


Figure 41. 41. MySQL service properties. Here you can reconfigure the service if you decide you don’t like how you set it up to begin with, including the name, description, and whether or not to start it automatically when the computer boots. You can also use this dialog to start and stop the service. The configuration file that MySQL uses during startup (my.ini) is also identified here; I’ll discuss how it works and how to change it in Chapter 7, “Configuring MySQL”. You should now be able to begin using MySQL.

Conclusion/Summary In this chapter, we installed MySQL on a computer running Microsoft Windows and learned a little bit about the plumbing underneath in order to troubleshoot potential problems. We’ll do the same in the next chapter for a computer running Linux, and then spend two chapters working with MySQL by itself and from Visual FoxPro. Updates and corrections to this chapter can be found on Hentzenwerke’s Web site, www.hentzenwerke.com.. Click “Catalog” and navigate to the page for this book. www.hentzenwerke.com

Chapter 4: Installing MySQL on Linux 55

Chapter 4 Installing MySQL on Linux This chapter supplements the on-line documentation with a detailed, step-by-step description of how to install MySQL from RPMs onto a Linux machine. It’s from the perspective perspective of an experienced Windows programmer who is new to Linux and provides background and perspective behind the steps. I’ll show you how to install MySQL on a SuSE Linux box, starting with downloading the right files, continuing with the installation, and finishing up with addressing potential problems. problems. Then I’ll discuss an alternative installation of MySQL on a Fedora Core 5/6 machine. Finally, I’ll describe what has happened as a result of the installation. installation. In subsequent chapters, I’ll describe the connection process between MySQL and VFP.

Installing MySQL on Linux these days is a reasonably straightforward procedure, and the online documentation is excellent – organized, detailed, and filled with a lot of examples. I urge you to navigate to http://dev.mysql.com/doc/refman/5.0/en/linux-rpm.html

for the official word. However, there are still a lot of variations you can encounter, and, as with any documentation team, there are still places where they have to assume a minimal level of knowledge and if you’re missing that knowledge, the procedure can still be difficult. In addition, as this is a book targeted toward VFP developers, it is very possible that you’ve not used Linux yet, but want to use this opportunity to do so. I’m going to assume you have a Linux machine running, but not much else, and I’ll review some of the basic concepts as we go along. At the same time, this chapter is not a tutorial on how to use Linux. (Fortunately, if you’re in need, there are a number of whitepapers and books available on the Hentzenwerke Publishing website that can help you get up to speed on Linux if you’re so inclined.)

Software and environment used This chapter was written using MySQL 5.0.18 and SuSE Linux 9.2, but I’ve used these same steps, with minor differences (such as version numbers of the downloaded MySQL files) to install other combinations, all the way back to MySQL 3.23 on Fedora Core 1.0. I vary between the GUI and the Command Window for various functions, but point out alternate methods when possible. In the examples used in this chapter, the Linux box is named ‘example’, with an IP address of 192.168.1.11 and a day-to-day user named ‘whil’. The user account in MySQL will be named ‘bob’. The Windows box running VFP that will connect to the Linux server has an IP address of 192.168.1.7.

The big picture – what installation does If you haven’t already, read Chapter 2 on how MySQL works to make sure you understand the context. MySQL on Linux consists, in a general sense, of two sets of files – the server and the client.

56


The server installation process installs the MySQL server – the actual software application. The Windows version of the MySQL server application itself is c:\mysql\bin\mysqld.exe c:\mysql\bin\mysqld.exe – on Linux, it’s called /usr/sbin/mysqld or /usr/libexec/mysqld (without any extension). The client installation puts a number of client programs and scripts (think ‘batch files’) in /usr/bin. The installation process then sets up the server as a service, enables the service to start automatically upon boot-up, creates user accounts and data files, and turns the keys over to you.

The essential installation process The basic process involves five steps: (1) Read the documentation on the MySQL site (2) Download the appropriate file(s), (3) Install the files, (4) Test the installation, and (5) Set passwords. There are several ways to install MySQL. The source distribution method involves unpacking gzipped ‘tar’ files (that have extensions of .tar.gz) while the binary distribution method uses RPM files and the RPM installer. This chapter describes how to install the RPM files on a SuSE (or compatible) system, using only the command window. There's also a brief description of installing MySQL on Fedora Core 5/6. It also assumes you have root access to the server and a regular user account. Some of you may be wondering about the choice of RPMs versus the other choices available. The reason we’re going to use RPMs instead of, for example, installing from the tar files, is that the RPM installation includes a number of scripts that are run automatically that set permissions, create databases, and set up the Linux ‘mysql’ user account. With the source distribution, you have to execute each of those steps yourself. While the source distribution option provides more flexibility and power, it’s probably p robably better to rely on the automated RPM installation your first time so you don’t miss any steps. Once you’re acquainted with MySQL to some extent, you can decide if it’s appropriate to try a source distribution installation. OK, there’s also another reason to use RPMs. My reason for using RPMs is that they work on a wide variety of distributions, are well-tested and reliable, and let’s face it, while it’s possible to install from source, why? In my best “Doctor McCoy” voice (the original Star Trek), “Dammit, Jim, I’m a software developer, not a system administrator!”

Read the documentation Since you’re reading this document, you’re not too adverse to reading the fine manual that comes with the product, and in this specific case, I urge you to take advantage of it. The online documentation is very good, and you shouldn’t take this document as a substitute for it. However, the documentation has to cover all possible scenarios, such as installing various types of files (RPMs and tar files) on Windows, Linux, and other operating systems, so it can be confusing having to skip parts that are not related to your particular situation. Thus, I winnowed out the parts not relevant to this particular type of installation, but point you to specific sections in the documentation when the time comes. So look at this document as a supplement to the on-line documentation, not a replacement.


Go to http://dev.mysql.com and and select the Documentation link. There are a number of options provided under the “MySQL Reference Manual” link. I’ve found the “HTML, viewable online” link, as shown in Figure 1, to be easy to navigate and very useful, as the user comments often provide additional tips and a variety of perspectives of real life usage.

Figure 1. 1. Link for on-line documentation. Clicking on the link will display a page that includes the index of the book on the left. Click the “Installing MySQL” link in that index and read away. The relevant sections (as of this writing – they occasionally change) you’re going to need, at the very minimum, are: 2.1.2 Choosing Which MySQL Distribution to Install 2.1.3 How to Get MySQL 2.1.4 Verifying Package Integrity Using MD5 Checksums 2.1.5 Installation Layouts 2.4 Installing MySQL on Linux

I mention these specifically because I’ll refer to each of these again in the following discussion, but don’t assume that you don’t need to read anything else in the on-line doc. Depending on your situation, you may find other sections useful as well.

Download the files Since this chapter assumes you’re going to install the RPMs, let’s get to it!

58


Decide on a download target You’ll download the RPM files after logging into your regular user account on your Linux machine. But before you actually download the MySQL files, you should decide where to put them. Personally, I put all the files I download into a directory called “zips” under my home directory, i.e. /home/whil/zips. (I also have a directory on the file server that serves a similar purpose. Once I find that I’m actually using a program or tool, I’ll archive the installation file(s) to the server for backup. It can be awfully inconvenient to want to reinstall something and find out you can’t get to the vendor’s website.)

Grab the files Once you have your own storage location decided upon, go to www.mysql.com www.mysql.com,, click on Developer Zone, Downloads, and find the “MySQL 5.0 – Generally Available (GA) release (recommended)” link as shown in Figure 2.

Figure 2 . Link for the Generally Available release. Scroll to the list of downloads for “Linux x86 generic RPM downloads” as shown in Figure 3. Click the server link – not the ‘max’.


Figure 3 . Links for Server and Client downloads. The next page you see will be a registration page (See Figure 4) with the identification of the file you’re downloading at the very top (easy to miss), like so: You are downloading MySQL-server-5.0.18-0.glibc23.i386.rpm

Figure 4 . Registration page displayed before selecting a mirror.

60


Note that, in Linux, this filename is case-sensitive, but that won’t matter in a moment. You have several options. You can Log In if you already have an account, fill out the registration form if you’re not registered, or just ignore the registration process by scrolling down below the form and selecting “No thanks, take me to the downloads.” In all of these cases, you’ll eventually get to the list of mirrors for your location. A sample if you live in the Midwest United States is shown in Figure 5.

Figure 5 . A list of Midwestern U.S. mirrors. Click on the link to download the file via HTTP or FTP as you desire. If you’re using Firefox, you can see the downloading progress as shown in Figure 6.

Figure 6 . The Firefox downloading dialog. (If you don’t see the downloading window, it probably means you turned it off in preferences. To open the window for this download, select Tools | Downloads in Firefox.)


Once the server download is complete, click back to the page shown in Figure 3, click on the “Client programs” link, and repeat the preceding few steps, culminating in another download window, shown in Figure 7.

Figure 7 . Downloading the client RPM file. If you followed my advice and created a ‘zips’ directory under your home directory, you can see a pair of files in the /home/whil/zips directory, where ‘whil’ should be replaced by your own Linux account name, as shown in Figure 8.

Figure 8 . Downloaded files located in your home directory’s ‘zips’ subdirectory.

62


You could also use the Command Window to list the files, like so: [whil@example: ~] cd ~/zips [whil@example: ~/zips] ls -al -rw-r--r-1 whil users 4814741 -rw-r--r-1 whil users 8568914

2006-01-04 MySQL-server-5.0.18-0.i386.rpm 2006-01-04 MySQL-client-5.0.18-0.i386.rpm

Note that the version (5.0.18.0 in this example) may change by the time you read this.

Verify the bits It’s a good idea to verify the integrity and authenticity of packages before installing them, as the bits can get scrambled while they’re being downloaded. See the on-line documentation’s section 2.1.4, “Verifying Package Integrity Using MD5 Checksums” for details.

Uninstalling an old version It is possible that you’ve made a mistake during these steps. (You really shouldn’t be trying to watch ‘Alias’ at the same time you’re installing MySQL. Trust me.) Or perhaps you’ve got an old version of MySQL on the box and you want to use the latest and greatest. Or for whatever other reason, you find yourself needing to remove MySQL from the box. There isn’t a single ‘magic button’ to do so, given the number of configurations that might exist on your machine, but here are a few tips to help. If you have an old version installed that came along with your Linux distribution – for example, if you clicked “database server” as one of the packages to install when you installed Linux – removing it is pretty simple. In SuSE, call up YaST, and enter ‘mysql’ in the Search textbox. Clicking the Search button will then display all packages that contain ‘mysql’ in the Name or Summary, which, given the richness of a typical Linux distribution, narrows down the field considerably. The packages that are installed are checked, as shown in Figure 9.


Figure 9 . The YaST tool displays which packages are available and which are installed. Working with this interface to uninstall a package isn’t completely intuitive. The initial expectation of most folks is to click on the checkbox next to a selected package, such as mysqld, expecting the action to uncheck the checkbox. But that’s not how it works. Instead, the icon will change to a circle with a jagged slash through it, as shown in Figure 10.

Figure 10 . Clicking the checkbox a single time doesn’t accomplish the desired ‘remove’ effect. The trick is to click a second time, so that the icon changes to a trash can, as shown in Figure 11.

64


Figure 11. Clicking on the icon a second time flags the package to be removed. Repeat this for each of the selected items you want to remove. Then click the Accept button in the lower right, and YaST will uninstall the components you’ve told it to. Depending on what you have installed on your machine, you may be presented with a screen that indicates the files you want to remove have dependencies elsewhere in the system. YaST is pretty good about explaining what your choices are for each occurrence, but it’ll ultimately be up to you to decide what to remove, what to keep, and what to ignore. Finally, YaST will finish the uninstallation, update the system configuration, and you should be all set to install 5.0 from scratch.

Install the files Now that you have genuine MySQL bits, switch to becoming the root user with the ‘su -’ command, like so: [whil@example ~/zips] su Password? [root@example /]

Since the ‘su’ (‘superuser’) command will change the default directory to root’s default, you’ll want to change back to the zips directory in your home directory [root@example /] cd /home/whil/zips [root@example /home/whil/zips]

Install the server Now, install the server, running the RPM commands as root. [root@example /home/whil/zips] rpm - i MySQL-server-5.0.18.0-i386.rpm

Remember my comment about the case sensitivity a few paragraphs back? If you’re new to Linux, you may be thinking that it’s going to be an awful nuisance having to type this long string exactly so, even if you’re used to VFP’s camelcase syntax. Here’s a trick that’ll save you heaps of typing – Linux has an ‘autocomplete’ feature that will do much of the work for you. After you type [root@example /home/whil/zips] rpm -i M


press the tab key. Linux will search the local directory for all files that begin with this string (“M”) and automatically complete the command as long as it finds a unique match. You’ll see the following: [root@example /home/whil/zips] rpm -i MySQL-

Linux stops here because there are two files that begin with this string (the ‘server’ and ‘client’ files). All you need to do is type enough letters to uniquely identify the rest of the string and hit tab again. In this example, just type ‘s’, hit Tab, and the rest of the file name will be filled out for you. Press Enter to execute the command. See? That case-sensitivity isn’t that difficult to deal with after all, is it? Remember, although you’re logged in as ‘root’, you’re accessing the RPM files from within your own home directory. The following listing shows what you might see when you switch to the root user and then run the install process. If you were me, that is. [root@example /home/whil/zips] su Password? [root@example ~] cd /home/whil/zips [root@example /home/whil/zips] rpm -i MySQL-server-5.0.18.0.i386.rpm warning: MySQL-server-5.0.18-0.i386.rpm: V3 DSA signature: NOKEY, key ID 5072e1f5 PLEASE REMEMBER TO SET A PASSWORD FOR THE MySQL root USER ! To do so, start the server, then issue the following commands: /usr/bin/mysqladmin -u root password ‘new-password’ /usr/bin/mysqladmin -u root -h indy password ‘new-password’ See the manual for more instructions. Please report any problems with the /usr/bin/mysqlbug script! The latest information about MySQL is available on the web at http://www.mysql.com Support MySQL by buying support/licenses at https://order.mysql.com Starting MysQL. [root@example ~/zips]

Now let’s get the client stuff installed.

Install the client Once you install the server, be sure to install the client package as well: [root@example /home/whil/zips] rpm -i MySQL-client-5.0.18.0.i386.rpm

If you run into an “error: can’t create transaction lock” error, like so: [whil@example ~/zips] rpm -i MySQL-server-5.0.18-0.i386.rpm warning: MySQL-server-4.0.20-0.i386.rpm: V3 DSA signature: NOKEY, key ID 5072e1f5 error: can’t create transaction lock

you may be trying to install without switching to root first. In this listing, you can see the command prompt shows ‘whil’ as the current user.

66


Set a password for the MySQL root user Unlike the Windows installation, Linux lets you do the install without forcing you to set a root password. While you can get away without setting the password for ‘root’ for the time being, you won’t get far in this book (or with this author) if you follow that path. Let’s get a password set up right away and then march on. As the Linux root user, type the following command: [root@example ~] /usr/bin/mysqladmin -u root password ‘new-password’

where ‘new-password’ is the password for the MySQL user named ‘root’. For example, if root’s password was topsecret

you’d type [root@example ~] /usr/bin/mysqladmin -u root password ‘topsecret’

I’ll explain how users, passwords, and permissions work in Chapter 5, “Configuration of Users and Hosts”.

Potential problems There are several types of problems that might occur during configuration. The most common involve firewall conflicts and collisions with prior installations.

Firewall conflicts Opening the firewall on the Linux box means making sure that port 3306 is open. On SuSE, for example, you can do this through the YaST Control Center – click on Security and Users on the left side, then on the Firewall applet. Select Reconfigure Firewall Settings, and then add ‘3306’ to the Additional Services list by clicking on the Expert... button. See Figure 14.

Figure 12 . Opening port 3306 in the SuSE firewall.


Installation with Fedora Core 5/6 An alternative method of installing MySQL on a Linux box is to use the version that comes loaded with the distribution. Most Linux distributions include MySQL as part of their packaging, and Fedora Core is no exception. (Neither is SuSE, for that matter.) You can either include MySQL as part of the original package choices, or install it as part of a software update later. In order to do the latter, select the Applications | Add/Remove Software menu option in FC5 or System | Add/Remove Software menu option in FC6. After a few moments (15 seconds? 30?), the list of installed software will be displayed in the Package Manager. Select the “Servers” node in the left hand list, and then click on the “MySQL” checkbox in the right hand list, as shown in Figure 13.

Figure 13 . Selecting the MySQL Server packages in the Fedora Core 5 Package Manager. If you like, you can drill down to see specifically which packages are automatically selected via the “Optional packages” button, as shown in Figure 14.

68


Figure 14 . Examining the specific packages selected for the MySQL Server. Once you’ve selected the packages you want, select the “Close” button, then the “Apply” button in Figure 13, and the application will be installed. You can double-check your work via the “List” button in Figure 13; 13 ; it will show you every package installed on the system. Scroll down to the area with names beginning with “my...”, as shown in Figure 15.


Figure 15 . Determining which packages have been installed on a Fedora Core system. Of particular interest are ‘mysql’ (the MySQL client programs), mysql-administrator (the native MySQL server manager that I’ll discuss in more detail in Chapter 7, “Configuring MySQL”), and mysql-server (the server itself). I would suggest that you not try to do a separate installation of MySQL from RPMs and then use the distribution’s installation process to attempt to override that standalone installation. The distribution-included version of MySQL can potentially conflict with the standalone version, and extracting your way out of that mess will take more time than its worth.

Installation results If installation succeeds, a number of things have happened.

MySQL files have been installed MySQL installs files in a number of directories. Which files are placed where depends on what type of installation is done. Details can be found in section 2.1.5, “Installation layouts,” of the on-line documentation. While not comprehensive, here is a list of files that you’ll find useful. /etc/my.cnf: the MySQL configuration file /usr/bin: Client programs and scripts

70


Some 20+ files, including mysql, mysqlcheck, mysqladmin, mysqld_safe, mysqlimort, mysql_install_db, and mysqlaccess. /usr/sbin: The mysqld server mysqld, mysqlmanager /var/lib/mysql: Log files, databases ibdata1, mysql.sock, test /var/lib/mysql/mysql: MySQL meta data databases user.myd, muser.myi, user.frm, db.myd/myi,frm, tables_priv.myd/i/frm, host.myd,myi,frm /usr/share/mysql: Error message and character set files dirs for various langs

Files installed on Fedora Core are in slightly different locations. /etc/my.cnf: the MySQL configuration file /usr/bin: Executables, scripts and client programs Some 20+ files, including mysqld_safe, mysqlcheck, mysqladmin, mysql_config, mysqldump, and mysqltest /usr/libexec: The mysqld server mysqld, mysqlmanager /var/lib/mysql: Log files, databases ibdata1, mysql.sock, test /var/lib/mysql/mysql: MySQL meta data databases user.myd, muser.myi, user.frm, db.myd/myi,frm, tables_priv.myd/i/frm, host.myd,myi,frm /usr/share/mysql: Error message and character set files dirs for various langs

For all practical purposes, though, the differences won’t matter. The concepts are all the same – the MySQL service will be started automatically (/etc/init.d/mysql) (/etc/init.d/mysql) and will look for the my.cnf configuration file in /etc, and the data files location is defined in the config file.

The Linux user account is created A Linux user account named “mysql’ is created, if it doesn’t already exist. This account is used for running MySQL but nothing else, and thus is set up to have limited rights. This is a user login account just like the one you use to log in to your Linux machine. In SuSE, you can see this user via the KUser applet found under the System | Configuration | KUser menu as shown in Figure 16.


Figure 16 . The KUser Manager shows the mysql user. In Fedora Core, the User Manager is found under Administration | Users and Groups. You normally don’t need access to the Linux mysql user account, but if you want to log on as that user for some reason, you’ll need to change the password to something you know by running the ‘passwd’ command as the Linux root user.

Data files are created The server RPM places data under the ‘/var/lib/mysql’ directory. Note that the Linux ‘mysql’ user has to have rights to this directory, or MySQL will get frustrated trying to start up. You can check whether the data directory can be accessed by the ‘mysql’ user by changing to the /var/lib/mysql dir, and then running running the ‘ls -al’ command, command, like so: [root@indy /var/lib/mysql] ls -al total 20580 drwxr-xr-x 4 mysql root 4096 drwxr-xr-x 20 root root 4096 -rw-rw---1 mysql mysql 10485760 -rw-rw---1 mysql mysql 5242880 -rw-rw---1 mysql mysql 5242880 -rw-rw---1 mysql root 1386 -rw-rw---1 mysql mysql 5 drwx--x--x 2 mysql root 4096 srwxrwxrwx 1 mysql mysql 0 drwxr-xr-x 2 mysql root 4096

Jul Jul Jul Jul Jul Jul Jul Jul Jul Jul

29 29 29 29 29 29 29 29 29 29

19:30 16:20 19:23 19:30 16:20 19:30 19:30 16:20 19:30 16:20

. .. ibdata1 ib_logfile0 ib_logfile1 indy.err indy.pid mysql mysql.sock test

The third column (right after the column with the numbers in it) lists the user who owns the file or directory in question. The RPM installation scripts automatically handle all of the ownership and permissions requirements. Details about what is required to perform these tasks should you want to do a manual install (such as creating grant tables, ownership, and what scripts are used to do so) are

72


covered in section 2.7, “Installing MySQL on Other Unix-Like Systems,” of the on-line documentation. Let’s take a quick look at what these data files are. MySQL can store data in two types of storage engines: MyISAM and InnoDB. The three files beginning with ‘ib’ are related to the initial InnoDB database created during installation. The .err file contains information about what happens during installation. You need to be root to access the file. For example, [root@example /var/lib/mysql] tail indy.err

will display the last 20 lines of the file. The .pid and .sock files are used on Unix systems. A PID is the server process ID, and the .pid file is the file in which the server writes its process. A Unix socket file is one way for a MySQL client to connect to the server. The ‘mysql’ directory holds the MySQL meta data – the databases for database definitions, users, hosts, and so on. The ‘test’ directory is where the test directory will be created if you use a MyISAM storage engine. We’ll visit these directories and files again.

Automatic startup entries are created The /etc/init.d directory contains the scripts used by the system during start up to initialize services or perform other tasks. The MySQL RPM installation process adds a ‘mysql’ entry in /etc/init.d. Links to the mysql script in /etc/init.d are made in the various /etc/rcN.d directories where “N” is a number between 0 and 6, each representing a specific runlevel. For example, on my machine, /etc/rc3.d contains a link named S90mysql while /etc/rc0.d contains a link named K20mysql. (Your machine may use different numbers, such as S80mysql or K10mysql.) The first script starts up MySQL in runlevel 3, while the second script shuts down MySQL when the shutdown procedure enters runlevel 0. More information on this is in section 2.9.2.2, “Starting and Stopping MySQL Automatically.” In most Linux distributions, you can also use a graphical tool to examine the services running on the machine. In SuSE, for example, you can use the SysV Init Editor (found under System | Service Configuration menu). The services available are shown in the far left list box and the services loaded during various run levels are shown in the smaller lists on the right. (Run Level 3 is when your machine boots to a command prompt, without a GUI, while Run Level 5 is when your machine boots and runs a GUI automatically.) See Figure 17.


Figure 17 . Displaying the MySQL service for runlevels 3 and 5 in SuSE.

The mysqld daemon is started If the RPM files that you install include MySQL-server, the mysqld server daemon should be up and running after installation. You can test to make sure that daemon is running by checking out the process listing. To do so, use the ‘ps’ command to see if there are ‘mysqld’ processes running (note the lagging ‘d’). [root@example ~] ps -aux

If you don’t see any entries like the following (note that these lines spill over onto two lines): root 3805 0.0 0.3 5972 980 pts/2 S 16:20 0:00 /bin/sh /usr/bin/mysqld_safe --datadir=/var/lib/mysql --pid-file=/va

74


mysql 3826 0.6 3.9 30360 10160 pts/2 S --basedir=/ --datadir=/var/lib/mysql --user=mysql mysql 3827 0.0 3.9 30360 10160 pts/2 S --basedir=/ --datadir=/var/lib/mysql --user=mysql mysql 3828 0.0 3.9 30360 10160 pts/2 S --basedir=/ --datadir=/var/lib/mysql --user=mysql mysql 3829 0.0 3.9 30360 10160 pts/2 S --basedir=/ --datadir=/var/lib/mysql --user=mysql mysql 3830 0.0 3.9 30360 10160 pts/2 S --basedir=/ --datadir=/var/lib/mysql --user=mysql mysql 3831 0.0 3.9 30360 10160 pts/2 S --basedir=/ --datadir=/var/lib/mysql --user=mysql mysql 3832 0.0 3.9 30360 10160 pts/2 S --basedir=/ --datadir=/var/lib/mysql --user=mysql mysql 3833 0.0 3.9 30360 10160 pts/2 S --basedir=/ --datadir=/var/lib/mysql --user=mysql mysql 3834 0.0 3.9 30360 10160 pts/2 S --basedir=/ --datadir=/var/lib/mysql --user=mysql mysql 3835 0.0 3.9 30360 10160 pts/2 S --basedir=/ --datadir=/var/lib/mysql --user=mysql root 3836 0.0 0.3 3724 772 pts/2 R

16:20 16:20 16:20 16:20 16:20 16:20 16:20 16:20 16:20 16:20 16:22

0:00 /usr/sbin/mysqld 0:00 /usr/sbin/mysqld 0:00 /usr/sbin/mysqld 0:00 /usr/sbin/mysqld 0:00 /usr/sbin/mysqld 0:00 /usr/sbin/mysqld 0:00 /usr/sbin/mysqld 0:00 /usr/sbin/mysqld 0:00 /usr/sbin/mysqld 0:00 /usr/sbin/mysqld 0:00 ps -aux

it means the mysqld daemon is not running. Immediately after installation, you may only have a couple of entries, like so: root /bin/sh /usr/bin/mysqld_safe --datadir=/var/lib/mysql... mysql /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql...

I mention this because you might have 50 or more processes running on your machine, and it can be easy to miss just a couple of mysql entries buried in the middle of the list. You should now be able to begin using MySQL.

Conclusion/Summary Setting up MySQL on a Linux machine is significantly different than doing so on Windows. But once the server is ready to accept hits, using it will be virtually the same as if it were on a Windows box. You’re just talking to a different IP address. In fact, I routinely administer and interactively use MySQL on a Linux box from tools installed on a Windows box, and those functions are the subject of the next chapters. Updates and corrections to this chapter can be found on Hentzenwerke’s Web site, www.hentzenwerke.com.. Click “Catalog” and navigate to the page for this book. www.hentzenwerke.com

Chapter 5: Configuration of Users and Hosts 75

Chapter 5 Configuration of Users and Hosts Once you confirm that the MySQL server is up and running, you need to do some configuration of users and hosts, and then test connecting to the server locally. The concepts and steps involved are the same for both Windows and Linux, so you need to read this chapter regardless of which operating system you’re using.

MySQL’s open source and Linux-y roots means it was designed to be worked from the command line. And its role as a full-grown database server (as opposed to the local data engine that comes with Visual FoxPro) means it has a built-in security model that requires you to become familiar with users and permissions from the start. This latter difference reminds me of moving from Windows 95 to Windows NT – from a wide open environment to one that’s by default locked down unless you purposely leave the door open. Let’s take a look at using the built-in command line, and then get comfortable with users and permissions.

Work with the server from the command line In this section, I’ll introduce you to some basic MySQL commands and scripts you can use to administer MySQL from a Windows DOS box or the Linux command prompt. The Windows examples will begin with a DOS-style path c:\

while the Linux examples will begin with /usr/bin

or a similar Linux-style path.

Windows and Linux command line tools To access the Windows DOS box, select Start | Programs | Accessories | Command Prompt. (I know, you’re wondering why I mention this – but for some folks, it might have been a long time since they last used a DOS prompt.) Use Alt-Enter if you want a full-screen window. In Chapter 3, Figure 30 shows a checkbox for “Include BIN directory in Windows PATH. If you did not select that checkbox, you will need to preface the names of the MySQL scripts with the path to where you installed MySQL (such as c:\mysql) and include the location of the scripts themselves (the ‘bin’ directory), like so: c:\> \mysql\bin\some_mysql_script

Or you can simply type commands like so:

76


c:> some_mysql_script

and the Windows PATH will know where to search for the command. In SuSE Linux and Fedora Core, select the K Menu (similar to the Start button in Windows) | System | Terminal.

Running MySQL scripts Windows users are generally spoiled in that they’ve been conditioned into thinking that any user has access to the entire machine. This means Windows users are used to opening a DOS box and running a command with nary a thought about whether the user they’re logged on as has the rights to run that command. (Which has made life for virus writers oh-so-much easier.) If you are one of those folks, you’re in for an awakening. When you run MySQL scripts, you have to provide the script with the appropriate credentials (i.e. a user account and a password that the MySQL engine knows about). If you don’t, your attempt at running the command will result in a message that looks something like this: error: 'Access denied for user 'abc'@'localhost' (using password: YES)'

I can almost guarantee that you’ll run into this message at one point or another, so now is an opportune time to talk about why it shows up and how to deal with it. When you run a command inside the Visual FoxPro IDE (or .NET or Java or whatever your fave is), the IDE is wide open. By running the IDE, you have proven to the environment that you have the rights to go about your business. When executing scripts or otherwise accessing the engine, you have to prove to MySQL that you have permission to do so. (There is an exception to this – if there are no passwords or other restrictions in the system tables, then anyone can run any command; but if you followed the installations instructions, that shouldn’t be the case.) You ‘fess up to MySQL by providing a MySQL user account and password along with the command you’re running. Thus, instead of simply typing c:\> some_mysql_command

you’ll tell MySQL that you’re including the username and you want it to prompt you for that account’s password, like so (remember, ‘bob’ is a MySQL user account): c:\> some_mysql_command -u bob -p

You don’t include bob’s password because it would be typed in clear text, and would be retained in your command history. Instead, you’ll be prompted for the password: c:\> some_mysql_command -u bob -p Enter password:

After entering the password, the command will execute. OK, now let’s try out some scripts.


Windows Important: When you installed MySQL, you created a MySQL root user account. (The ‘root’ account is similar to the ‘sa’ account for Microsoft SQL Server.) In Windows, this was done in Figure 31 of Chapter 3. The password you entered in that figure is what you’ll use (along with the ‘root’ user) when running commands in the next few examples, like so: c:\> some_mysql_command -u root -p

Linux In Linux, this was done in the “Install the server” section of Chapter 4. However, in contrast to the Windows installation, the Linux installation doesn’t force you to create a password for the root user – indeed, the completion of the installation reminds you in capital letters to set a password. If you were a good reader and followed instructions, the password you used in the “Set a password for the MySQL root user” section in Chapter 4 is the password you’ll use, like so: [root@example ~] /usr/bin/some_mysql_command -u root -p

As a reminder, the ‘root’ in the command prompt (root@example) is the Linux user named ‘root’, while the ‘root’ user that follows the ‘-u’ flag is the MySQL root user account. The Linux root user is who is logged into the machine, while the MySQL root user is the user who can get into the MySQL server. We’ll create a separate MySQL user account (‘bob’) for day-to-day use later in this chapter and hopefully end this confusion.

Windows and Linux By the way, if you didn’t set a password, this whole discussion is moot – you’ll be able to run scripts without telling MySQL which user account and password to use. And so will everyone else in the world, if you get my drift.

Use mysqladmin to see if the service or daemon is running The MySQL client installation includes a tool called “mysqladmin” that is used for what it sounds like – administering MySQL. Obviously, you need to have the client tools installed in addition to the server to have mysqladmin available, a requirement determined the hard way by yours truly when, in the heat of the moment, he forgot to do the client install after the server install. If you forgot as well, the error you get will look something like error: 'mysqladmin not found'

In Windows, run the following command (assuming you installed MySQL in c:\mysql): c:\>

\mysql\bin\mysqladmin version –u root -p

In Linux, issue the following: [whil@example ~] /usr/bin/mysqladmin version –u root -p

78


Both should generate feedback similar to that shown in the following listing (this is for the Linux version). /usr/bin/mysqladmin Ver 8.40 Distrib 5.0.18, for pc-linux on i686 Copyright (C) 2000 MySQL AB & MySQL Finland AB & TCX DataKonsult AB This software comes with ABSOLUTELY NO WARRANTY. This is free software, and you are welcome to modify and redistribute it under the GPL license Server version Protocol version Connection UNIX socket Uptime:

5.0.18-standard 10 Localhost via UNIX socket /var/lib/mysql/mysql.sock 4 hours 35 min 51 sec

Threads: 1 Questions: 23 Slow queries: 0 tables: 2 Queries per second avg: 0.001

Opens: 8

Flush tables: 1

Open

As you can see, the two commands are identical, except for the OS prompt. From now on, I’m not going to provide duplicate Windows and Linux examples of each command unless they’re different.

See what MySQL variables are available MySQL has a number of system variables that contain useful information about itself and how it interacts with the environment. Using the ‘variables’ parameter with the mysqladmin command generates a long, long list of these variables. A few of these variables are shown in the next listing. [whil@example ~] /usr/bin/mysqladmin variables -u root -p +---------------------------------+---------------------------------------+ | Variable_name |Value | +---------------------------------+---------------------------------------+ | back_log | 50 | | basedir | / | | binlog_cache_size | 32768 | | datadir | /var/lib/mysql/ | | default_week_format | 0 | | pid_file | /var/lib/mysql/example.pid | | port | 3306 | | socket | /var/lib/mysql/mysql.sock | | version | 5.0.18-standard |

Take a few moments and read through the list of variables. While some will be fairly obscure, others will make immediate sense. I’ll cover some of these in Chapter 7 on “Configuration.”

Stopping and starting the server – Windows There are several related concepts necessary to understand when working with the MySQL server in a Windows DOS box.

Server versus service You can get MySQL ‘running’ either as a server application or as a service in the operating system. Details on HOW for both shortly.


The usual method is to have the operating system automatically start the MySQL server by setting it up as a service through the Services applet. This means that the MySQL server is available all the time (well, as long as the machine is on). It’s chewing up resources such as RAM and processor cycles, but for a live application, you’ll typically want the database available 24/7. The alternative is to manually start the MySQL server as a program (say, by clicking on a desktop shortcut or in a DOS box), and then shut it down by exiting the program. This method means the MySQL server is only available for as long as the program is running, much like any other program (Visual FoxPro, Paint Shop Pro, Firefox). If you’re low on resources, you may choose this route to conserve those valuable processor cycles and bits of RAM. You can get away with this method during development, but not for a production application. However, starting and stopping the MySQL server from a DOS box isn’t the same as doing it from the Services applet. In particular, you shouldn’t mix the two – don’t start it up through the Services applet and then try to shut it down through commands issued in the DOS box, or vice versa. Doing so can hang up the service or, possibly, the machine itself. In the following discussion, you’ll want to make sure the MySQL service is not running before trying commands in the DOS box. Open the Services applet under Administrative Tools in the Control Panel, right-click on the MySQL service, and select the “Stop” item in the context menu.

The Windows MySQL binary MySQL provides a special binary executable for Windows, called mysqld-nt.exe, which is optimized for the Windows operating system. (You can also run the standard ‘mysqld.exe’ binary, but it’s not quite as fast.) When MySQL is running as a service in Windows (as described at the end of Chapter 3), it’s running mysqld-nt.exe, and you can see this name listed in the Processes pane of the Task Manager (right-click in the Quick Launch toolbar at the bottom of the screen in Windows, and then select “Task Manager” from the context menu).

Starting and stopping as a service You can use the NET command to start and stop any service, including MySQL. (Remember to make sure that MySQL isn’t already running!) c:\>net start mysql

and then stop it like so: c:\>net stop mysql

Starting and stopping in the DOS box You can also start and stop the server in the DOS box instead of using the NET command. However, you don’t want to mix starting in one place and stopping in the other. So, again for the purposes of this discussion, make sure MySQL isn’t running! Now we can start the server in the DOS box and shut it down again. Make sure the server process is gone from the Task Manager. To start the server in the DOS box, issue the command:

80


c:\> \mysql\bin\mysqld-nt

You’ll see ‘mysqld-nt’ show up again in the Task Manager. The tricky thing here is that the server will continue to write output (if any) to the DOS box. In other words, after issuing the mysqld-nt command, the DOS box does not relinquish control back to you. You will need to open a new DOS box to issue more commands. To shut down the server, open a new command prompt and issue the command: c:\> \mysql\bin\mysqladmin -u root -p shutdown

You’ll see the mysqld-nt entry in the Task Manager disappear in a few moments. If you’ve been following the steps so far, you’ll want to restart the service (either via the NET command or via the Services applet) so you can continue later in this chapter.

Stopping and starting the server – Linux Starting and stopping the server in Linux is easy. Linux’s mission in life, if you want to think of it that way, is to be the mom for a collection of services.

Stopping the server First, make sure you can shut the server down. (Restarting will be covered next.) As either a regular or root user, issue the mysqladmin command as shown here: [whil@example ~] /usr/bin/mysqladmin -u root -p shutdown

The results should look something like the following, [whil@example ~] 040731 16:57:24

mysqld ended

although on my SuSE box, I just get a blank prompt. This test is also covered in section 2.9.2, “Unix Post-Installation Procedures,” in the online documentation. You can now do [whil@example ~] ps -aux

and note that the mysql entries are gone.

Restarting the server It’s darn handy to be able to restart the server after you shut it down. The mysqld_safe command - run as the Linux root user - does this as shown. (Note that there’s a ‘d’ after ‘mysql’ in the command!) First, switch to root with the ‘su -’ command. Then issue the mysqld_safe command. Finally, switch back to your regular user with the ‘exit’ command. [whil@example [root@example [root@example [whil@example

~] su ~] /usr/bin/mysqld_safe --user=mysql --log & ~] exit ~]


Let’s take a look in more detail at the command that starts up the server. The “mysqld_safe” part is the name of a script that runs the mysqld command that starts MySQL. The first parameter, --user=mysql, identifies the Linux user whose account will be used to run the mysqld command. The second parameter, --log, logs connections and queries to the default log file located in /var/lib/mysql, which is particularly useful when you’re getting started and might need to do some troubleshooting. The ampersand, “&”, is technically not part of the mysqld_safe command, but rather is a standard Linux shell command metacharacter. The ampersand tells the issued command to run as a background process so the command window comes back. It’s like the ‘nowait’ clause found in Visual FoxPro. If you don’t use the “&” at the end, the mysqld_safe process will run in the foreground and remain ‘tied’ to the command window you started it from. So if you close the command window, you may kill the mysqld_safe process. As noted earlier, mysql creates several kinds of files in the /var/lib/mysql directory, including a .err, a .pid, and a .sock file. If you attempted to start mysql while logged onto the machine as a Linux user who doesn’t have rights to that directory, your attempt will fail, and you’ll get an error like this: /usr/bin/mysqld_safe --user=mysql --log & Starting mysqld daemon with databases from /var/lib/mysql /usr/bin/mysqld_safe: line 308: /var/lib/mysql/example.err: Permission denied /usr/bin/mysqld_safe: line 1: /var/lib/mysql/example.err: Permission denied tee: /var/lib/mysql/example .err: Permission denied 040731 17:21:09 mysqld ended tee: /var/lib/mysql/example.err: Permission denied [1]+ Exit 1 /usr/bin/mysqld_safe --user=mysql --log

Once you log in with sufficient rights, you’ll see something like the following listing if your startup is successful. [root@example ~] /usr/bin/mysqld_safe --user=mysql --log & [1] 4245 [root@example ~] Starting mysqld daemon with databases from /var/lib/mysql [root@example ~]

The number echoed in response to the command, in this case 4245, is the process ID. You can see it if you run the ps -aux command. This test is also covered in Step 6 of section 2.9.2, “Unix Post-Installation Procedures,” in the on-line documentation. Note that section 5.7.5, “How to Run MySQL as a Normal User,” of the on-line manual states, “On Unix, the MySQL server mysqld can be started and run by any user. However, you should avoid running the server as the Unix root user for security reasons. In order to change mysqld to run as a normal unprivileged Unix user user_name, you must do the following...” This can be confusing, in that if you try to execute the script that runs mysqld, like so: /usr/bin/mysqld_safe

as a regular Linux user (i.e. not as the mysql Linux user), you’ll get the “Permission denied” error mentioned earlier. What the paragraph from section 5.7.5 is saying is that you need to start the script as the Linux root user, but the script should tell mysqld to run as a regular Linux user. One such

82


regular Linux user would be our fallback user, ‘bob’; another example of a regular Linux user would be the ‘mysql’ user the RPM installation process creates. The (Linux root usercontrolled) script gets the MySQL daemon running. The daemon then runs under a regular Linux user account, as I demonstrated earlier with the ‘ps -aux’ command.

More useful MySQL scripts At this point (if the previous tests have been successful), the MySQL service/daemon is running. There are a number of commands you can issue to MySQL via the operating system, similar to the mysqladmin command. For example, /usr/bin/mysqlshow -u root -p

will display a block character graphic of the databases available to MySQL, like so: [whil@example ~] /usr/bin/mysqlshow -u root -p +--------------------+ | Databases | +--------------------+ | information_schema | | test | +--------------------+

If you’re logged into your Linux machine as a regular user, you’ll only see the test database (and any others that have wide open permissions, which is hopefully none). If you’re logged into your Linux machine as root, however, you’ll see all of the databases, like so: [root@example ~] /usr/bin/mysqlshow +--------------------+ | Databases | +--------------------+ | information_schema | | mysql | | test | +--------------------+

-u root -p

Including the name of a database name as a parameter to the mysqlshow command will display the tables in the database. [root@example ~] /usr/bin/mysqlshow mysql -u root -p Database: mysql +---------------------------+ | Tables | +---------------------------+ | columns_priv | | db | | func | | help_category | | help_keyword | | help_relation | | help_topic | | host | | proc |


| procs_priv | | tables_priv | | time_zone | | time_zone_leap_second | | time_zone_name | | time_zone_transition | | time_zone_transition_type | | user | +---------------------------+

Another handy command is ‘mysql’. With it, you can even execute a single SQL command, as in this example: [root@example ~] /usr/bin/mysql -e "select host,user from mysql.user" -u root p +------------+--------+ | host | user | +------------+--------+ | localhost | root |

If you don’t get output from this type of command, it means the table is empty.

Summing up We used a number of command line commands and it can be confusing to remember what each one is for. Here’s a quick summary: MySQL – the MySQL server on Windows, optimized for InnoDB support. MySQLD – the MySQL server on Linux. MySQLShow – client for displaying database information (which databases exist and the metadata for a database). MySQLAdmin – client for administering a MySQL Server. MySQL-NT – the MySQL server on Windows, for Windows NT, 2000 and XP, with support for named pipes.

The MySQL interactive environment (MySQL monitor) If all of these tests went well, you can pretty much assume that MySQL is up and running and ready for you to take charge. The next thing you’ll want to do is get into the MySQL interactive environment, called the MySQL monitor. This environment offers a command prompt that allows you to interact with MySQL much like the Windows DOS box or the Linux command prompt allows you to issue commands to the Windows or Linux operating system. If you’re used to manipulating VFP tables from the VFP Command Window, you’ll be right at home with the MySQL monitor.

Enter the MySQL interactive environment While the monitor will work roughly the same in both operating systems, the way you enter the MySQL monitor is slightly different.

Windows The process to load the MySQL monitor in Windows varies according to whether the server is already running. If you don’t have MySQL running (either as a service or as a server), open a

84


DOS box, switch to the ‘bin’ directory under the MySQL installation directory, and execute the mysqld command, as shown in the following listing: c:\mysql\bin> mysqld –-console (some text) 060323 18:51:11 [Note] mysqld: ready for connections. Version 5.0.15' socket: '' port: 3306 Official MySQL binary

Note that there are two hyphens before the word “console” in the command! You’ll notice that control doesn’t return to you – the DOS box seems to hang. This is because the server is running in the box, and can continue to send output to the DOS box as it needs to. Thus, you’ll need to open a second DOS box to connect to the monitor. Now that you have the server running, it’s time to connect to the monitor itself, like so: c:\mysql\bin> mysql -u root -p password: Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 6 to server verrsion: 5.0.21-community-net Type 'help;' or '\h' for help. Type '\c' to clear the buffer. mysql>

The ‘mysql>’ prompt is our desired result. It provides access to the MySQL engine. If you’re not interested in Linux, you can skip the next section and continue with “Exit the MySQL interactive environment.”

Linux In order to load the MySQL monitor in Linux, issue the command: [whil@example ~] mysql -u root -p

and you’ll get a new prompt in your command window, like so: Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 7 to server version: 5.0.18-standard-log Type 'help;' or '\h' for help. Type '\c' to clear the buffer. mysql>

At this point, you’re logged into the MySQL environment as the MySQL root user (not the Linux root user.)

Exit the MySQL interactive environment Once you’re in, you need to know how to get out. I know, you’re a Windows user – you know the answer – just Ctrl-Alt-Del or kick the cord out of the wall...

Windows If you’re running Windows, type ‘quit’ or ‘exit’ and press Enter. You’ll be returned to the second DOS box. In order to shut down the server, issue the mysqladmin command with the


‘shutdown’ parameter as described earlier in this chapter. Control will be returned to you in the first DOS box, like the last two lines in the following listing (shown in bold): c:\mysql\bin> mysqld -–console (some text) 060323 18:51:11 [Note] mysqld: ready for connections. Version 5.0.15' socket: '' port: 3306 Official MySQL binary 060323 19:15:59 [Note] mysqld: Normal shutdown 060323 19:16:01 [Note] mysqld: Shutdown complete

Linux If you’re running Linux, you just need to type ‘quit’ or ‘exit’, and press Enter. mysql> quit Bye [whil@example ~]

You’ll be returned to the command prompt as shown. As you’ll see in a moment, commands typed into the MySQL monitor generally need to be terminated with a semi-colon. The ‘quit’ and ‘exit’ commands are why I say ‘generally’ – you don’t need to type a terminating character with either of those commands.

A quick tour of the MySQL monitor A few introductory instructions about the MySQL monitor are in order. First, note that you loaded the monitor while logged into the machine as a regular user (whil). However, the “-u root -p” clause after the ‘mysql’ command logs you into MySQL as the root user. If there was a second user already set up in MySQL, such as ‘bob’ (in addition to ‘root’), you could have issued the command [whil@example ~] mysql -u bob -p

and you would have been able to maneuver around inside of MySQL with all of the rights and permissions that ‘bob’ has been granted inside MySQL. Again, ‘bob’ has no context on the machine outside of MySQL. Second, you need to terminate a command with the semi-colon command (;) or \g in order to get MySQL to react to it. This means you can type a command on multiple lines, like so: mysql> select NameFirst, NameLast -> from customers -> where State = "WI" ;

If you hit the Enter key at the end of a line without first typing one of the terminating characters, MySQL will provide a new line with a “->” prompt and the cursor will simply jump to the beginning of the line. (Readers who are familiar with Visual FoxPro’s syntax of using the semi-colon as a line continuation character will surely be driven mad by MySQL’s reverse behavior, but C, C++, and C# developers are already used to it. I guess that’s why we’re all paid the big bucks.) Type “Help” at a mysql> prompt for more information about the MySQL monitor.

86


Set passwords and new accounts It is possible that the work we’ve done so far has been without the benefit of passwords on the accounts set up inside of MySQL. At the very least, it has been done as the root user inside MySQL. Yikes! The next step, now that you’ve confirmed you can start and stop the MySQL server and interact with it, is to set up passwords and create work-a-day MySQL users. If you’re working on a test server behind a firewall, you may be tempted to wait for a bit before doing these chores, but I’d encourage you not to. It can be rather frustrating to spend a lot of time on a development project while using a test database without permissions or users set up, or to do the bulk of your work as ‘root’, only to find that you have to spend significant time reworking things because your system doesn’t work like you thought it would when operated by regular users. The first time you start the MySQL server, you’re prompted like so: PLEASE REMEMBER TO SET A PASSWORD FOR THE MySQL root USER ! To do so, start the server, then issue the following commands: /usr/bin/mysqladmin -u root password 'new-password' /usr/bin/mysqladmin -u root -h example password 'new-password' See the manual for more instructions. Please report any problems with the /usr/bin/mysqlbug script! The latest information about MySQL is available on the web at http://www.mysql.com Support MySQL by buying support/licenses at https://order.mysql.com

You’ll also read in the documentation that you can set passwords inside MySQL (after you enter the MySQL monitor) with the ‘set password’ command and the password function. However, what’s happening behind the scenes can be obscure for the new MySQL user. Here’s a little background, but first, let’s add a brand new user with a new password so you have a test user to monkey around with.

Quickly adding a user We’ll add a user named ‘herman’, with a password of ‘bigsecret’, only for the localhost host. First, get to the MySQL monitor prompt. I’ll use the Windows version but the commands are the same for Linux. mysql>

Issue the following command mysql> GRANT ALL PRIVILEGES ON *.* TO 'herman'@'localhost' IDENTIFIED BY 'bigsecret' WITH GRANT OPTION;

To verify that you added the user properly, issue the following SQL SELECT command to display all accounts: mysql> select host, user, password from mysql.user;


You should see a display like so: +-----------+--------+------------------+ | host | user | password | +-----------+--------+------------------+ | localhost | root | 38914f9560d4d960 | | localhost | herman | 73502e3264b4a950 | +-----------+--------+------------------+ 2 rows in set (0.07 sec)

Finally, quit the MySQL monitor with ‘quit’ or ‘exit’: mysql> quit

Now that good ol’ herman is ready for testing, let’s discuss the concepts underlying these operations.

MySQL password concepts As you know, when you log into a machine (be it Windows, Linux, Mac or one of those DEC PDPs from the last century), you need to provide two credentials: a username (also called an ‘account’ or ‘account name’) and a password. Some systems will let you create an account without a password, but most don’t. When you log into MySQL, you need to provide three credentials: a username (again, also referred to as ‘an account’), a password (also optional, but lack thereof highly discouraged), and a third item – the host value. The host value is the name or the IP address of the machine that the user is connecting from. In other words, suppose your MySQL server is on a box with the hostname of “serverbox.” Further suppose you have a workstation whose hostname is “userbox,” and you have logged into “userbox” with the username of ‘herman’ and the password of ‘bigsecret’, you would need to create an account in MySQL with the credentials: username: herman password: bigsecret hostname: userbox

As a result, you can restrict access not only to specific users, but to specific users who are connecting from specific hosts. Thus, you could let user ‘herman’ connect from his home machine with a specific IP address, but not let him connect from anywhere else. You could also use a wildcard for a range of IP addresses, like so: 192.168.1.%

This would allow a user to access from all of the hosts on a single subnet. These three attributes – username, password, and hostname – are contained in a MySQL system table that looks (partially) like this: +------------------+--------+------------------+ | host | user | password | +------------------+--------+------------------+ | userbox | herman | 48bf4fd20c61a2f0 |

88


| another_box | herman | 48bf4fd20c61a2f0 | +------------------+--------+------------------+

Thus, when you’re setting passwords, you’re updating the password field for a host/user combination. When you’re adding MySQL users, you’re adding rows to this table. With that background, let’s revisit the mysqladmin command we used in Chapter 4 to set the MySQL root user password. [root@example ~] /usr/bin/mysqladmin -u root password 'topsecret'

As you can see, we specified the username, ‘root’, and a password for ‘root’ – ‘topsecret’. However, you’ll notice there’s no mention of a host anywhere. This is because, in the absence of an explicit hostname declaration, mysqladmin will assume the local machine’s server, localhost: username: root password: topsecret hostname: localhost

If you wanted to use mysqladmin to set root’s password on another host, you’d include the hostname with the ‘-h’ flag, like so: [root@example ~] /usr/bin/mysqladmin -h somehost -u root password 'topsecret'

The mysqladmin command is only for setting the root user’s password. You can use the ‘set password’ command and ‘password’ function to manipulate the root user’s password as well as to set other user’s passwords. (You can also use the update command, but... one thing at a time, please.) Using ‘set password’ is done like so: mysql> set password for ''@'localhost' = password('newpassword');

The password function, password(‘newpassword’), encrypts the password before storing it in the database. The syntax of the part to the left of the equal sign, though, can be confusing at first. The first set of empty quotes, before the ‘@’ sign, normally contains the name of the user. The Linux RPM installation routine automatically creates two users in the MySQL user database. One is root and the other has no name – it’s the anonymous user, which is represented by an empty string: ‘‘. (That’s two single quotes, without anything in between them.) In Windows, the checkbox in Figure 31 in Chapter 3 allows you to decide whether or not you want to create an anonymous user along with the root user. If you wanted to set the password for root using this syntax, you’d use mysql> set password for 'root'@'localhost' = password('newpassword');

Correspondingly, if you want to set a password for the anonymous user, you simply don’t enter a username, and you get ‘‘@’localhost’: mysql> set password for ''@'localhost' = password('newpassword');


You can find out what users exist and whether passwords, if any, exist for those users, with a SQL SELECT statement, like so: [whil@example ~] mysql -u root -p mysql> select host, user, password from mysql.user ; +------------------+------+------------------+ | host | user | password | +------------------+------+------------------+ | localhost | root | 48bf4fd20c61a2f0 | | example | root | 48bf4fd20c61a2f0 | | localhost | | | | example | | | +------------------+------+------------------+ 4 rows in set (0.07 sec)

This shows that the MySQL user, ‘root’, has had a password assigned, but the anonymous user has none. The password shown, 48bf4fd20c61a2f0, by the way, isn’t the real ‘root’ password, but the hashed version, due to the use of the password function in the ‘set password’ command. You’ll want to set passwords for both the localhost and real name domains, both for root and the anonymous user. The following commands use the word ‘secret’ for the password for all four user/host combinations: mysql> mysql> mysql> mysql>

set set set set

password password password password

for for for for

''@'localhost' = password('secret'); ''@'example' = password('secret'); 'root'@'localhost' = password('secret'); 'root'@'example' = password('secret');

I’d suggest you don’t actually use the word ‘secret’ as your password, though. This information is covered, with a lot of additional examples, in section 2.9.3, “Securing the Initial MySQL Accounts.” Definitely worth checking out!

How MySQL user passwords affect commands I want to briefly mention that you need to include the “-u” and “-p” flags when running MySQL scripts. Now that your feet are wet with users, passwords, and hosts, it’s probably a good time to revisit this concept. After you assign a password to the root mysql user, you’ll need to prompt for a password with the -p clause as well, like so: [whil@example ~] /usr/bin/mysqladmin -u root -p shutdown Enter password: [whil@example ~] 040731 16:57:24 mysqld ended

Or else you’ll get an error like so: [whil@example ~] /usr/bin/mysqladmin -u root shutdown /usr/bin/mysqladmin: connect to server at 'localhost' failed error: 'Access denied for user: 'root@localhost' (Using password: NO)'

Another example of failure is when you try to run the MySQL monitor:

90


[whil@example ~] mysql -u root ERROR 1045: Access denied for user: 'root@localhost' (Using password: NO)

Including the ‘-p’ clause results in this exchange with the computer: [whil@example ~] mysql -u root -p Enter password: Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 11 to server version: 4.0.20-standard-log Type 'help;' or '\h' for help. Type '\c' to clear the buffer. mysql>

And now you’re connected to the MySQL database server via the monitor as the MySQL user named ‘root’. If you’re trying to connect with a MySQL server on a separate machine, you may need to include the “-h” flag, like so: [whil@example ~] mysql -h some_host -u root -p

or like so: [whil@example ~] mysql -h www.another_sample_domain.com -u root -p

If your local machine is set up so it can resolve “some_host,” you won’t need the “-h” flag. If, however, you get the following: [whil@example ~] mysql another_host -u root -p Enter password: ********** ERROR 1049 (42000): Unknown database 'another_host'

you may need to explicitly identify the host to connect to via the “-h” flag. Note: If you install MySQL on Linux using RPM distributions, the server RPM runs the mysql_install_db script automatically. This script creates the mysql and test databases and sets up initial users. If you install differently (say, from source – blech!), you’ll need to do this yourself. See “Unix Post-Install Procedures” in the on-line help for step-by-step instructions.

Adding a work-a-day user Just like you wouldn’t want to use MySQL with a password-less user, you won’t want to connect to MySQL via the root user for your regular daily work. Time to create a ‘work-a-day’ user. Remember ‘bob’? GRANT ALL PRIVILEGES ON *.* TO 'bob'@'localhost' -> IDENTIFIED BY 'secret' WITH GRANT OPTION;

This will create a user named ‘bob’ with the password of ‘secret’ who has full privileges on localhost. You can now access MySQL like so:


[whil@example ~] mysql -u bob -p Enter password: mysql>

Remember the mysql> select host, user, password from mysql.user ;

command in order to look at what users (and their permissions) are in your system. The mysql.user table actually has a lot of fields in it, each of which specifies permission for a specific action. For example, the contents of the ‘select_priv’ field determines whether a given host/user combination can do queries, while the contents of the ‘insert_priv’ field determines whether or not a host/user combo can insert records.

Conclusion/Summary Getting used to accessing a back-end database takes a bit of time, but after spending an hour or two practicing, you’ll feel right at home. MySQL’s user access architecture is straightforward and easy to work with. However, if you are not a command line guru, and would rather deal with one of the GUI interfaces for all of your administration, you are in luck. Regardless of the OS, there is both a Web interface and a GUI interface that allows you to manage users, add and grant privileges, and create a database and tables without ever seeing a command line. It’s nice to know the command line is there if you want it, but it’s not necessary as you’re building your VFP-MySQL applications. Let’s move on to connecting these tools. Now it’s time to connect to MySQL from Visual FoxPro. Updates and corrections to this chapter can be found on Hentzenwerke’s Web site, www.hentzenwerke.com. Click “Catalog” and navigate to the page for this book.

92


Chapter 6: Connecting VFP to MySQL 93

Chapter 6 Connecting VFP to MySQL Now that we have MySQL server up and running with the right users and hosts, it’s time to direct our attention to the Windows box running VFP. In this chapter, we will connect to MySQL from within Visual FoxPro, using the MySQL ODBC driver and a brief set of VFP commands.

This chapter covers two processes – the installation of the ODBC driver, and the subsequent use of the driver from within VFP to connect to the MySQL server. In this chapter, I use multiple screen shots of the ODBC driver installation process. In the past, the screens have changed from one version to another, so it’s possible that they have changed again since this writing. I’ve found, though, during the review process, that the end result of the ODBC installation is the same, and you shouldn’t have any trouble if you’re using a later version than what is described in this text. After a quick stop to install the ODBC driver, we’ll go right to the interesting stuff – connecting from VFP.

Installing the MySQL ODBC driver Now that MySQL is installed and you have confirmed that it is working, it’s time to address the next piece of the puzzle – the connector between VFP and MySQL: the ODBC driver.

Uninstalling an old version of the MySQL ODBC driver If you need to uninstall a previous version of the MySQL ODBC driver, you can do so through the Add/Remove Programs applet in the Control Panel. Scroll down the list and look for “MyODBC” in the list of programs.

Downloading the MySQL ODBC driver Head back to the MySQL download page and grab the MySQL ODBC driver. Click on the Connectors link in the same band as the “Database Server” link, and follow the same procedure for either the Driver Installer (Zipped EXE) or Driver Installer (MSI) link. Depending on which one you download (they’re functionally the same) the file will be named something like mysql-connector-odbc-3.51.12-win32.zip

or mysql-connector-odbc-3.51.12-win32.msi

They’re both about 3 MB in size.

94


Installing the MyODBC driver Either unzip the ZIP file and run the EXE file that it contains, or right-click on the MSI file and select the “Install” option in the context menu. In both cases, the wizard will start, as shown in Figure 1.

Figure 1. Starting the MySQL ODBC driver wizard. Continue through the next couple of dialogs, and then choose the “Custom” selection shown in Figure 2’s setup type selection.


Figure 2 . The set up type selection for the ODBC driver. I always choose Custom when doing installs so I can see what the various features are. Often I go with the defaults, but it’s nice to see what else might be available. This brings us to the Custom Setup screen shown in Figure 3.

96


Figure 3 . The feature selection tree for the ODBC driver. In this particular situation, I keep all the features selected. Click “Next” to begin the installation, shown in Figure 4.


Figure 4 . Starting the installation of the ODBC driver. You’ll note that there is no installation folder selection – ODBC drivers are all handled the same way so that Windows knows where to find them. The thermometer bar and progress messages will display as expected and shown in Figure 5.

Figure 5 . The thermometer bar and message display during installation.

98


And if all goes well, you’ll get the confirmation dialog as shown in Figure 6.

Figure 6 . The confirmation dialog for the ODBC driver. Open the ODBC Data Source Administrator. This applet is found via Start | Settings | Control Panel | Administrator tools | Data Sources (ODBC) in Windows 2000, and Start | Control Panel | Performance and Maintenance | Administrator Tools | Data Sources (ODBC) in Windows XP. Note that some configurations of XP do not have a “Performance and Maintenance” folder. Click on the Drivers tab. You’ll see the MySQL ODBC driver in the list along with whatever other drivers are installed on your computer, as shown in Figure 7.


Figure 7 . The Data Source Administrator dialog now shows the MySQL driver too. You can use this same driver multiple times, with multiple versions of Visual FoxPro and/or MySQL – you don’t have to install multiple copies for different purposes. The installation places the myodbc3.dll and myodbc3.lib files in the WinNT\system32 directory.

Connecting to MySQL from VFP We’re going to test the connection from VFP to MySQL in several steps. First, we’ll do it the ‘wizard’ way – by using a DSN (“Data Source Name”) that hides the details of the connection from us. This step will help us verify that the connection works, since error messages that spring up during the use of a DSN are easier to interpret than when testing manually. Next, we’ll manually build a connection string from the DSN, and use that string to do the same connection to VFP. Along the way, we’ll learn the basics of issuing commands to the MySQL database server from within VFP. We’ll also look at how to use VFP’s error handling capabilities to trap errors and what those errors mean to us in the real world.

Connecting to MySQL from VFP with a DSN DSNs are Windows objects that contain the information necessary to make a connection to a data source. These objects are stored in the Windows Registry. There are three types: File, User, and System.

100


A File DSN can be stored anywhere and used by any user who has access to the appropriate ODBC drivers. A User DSN, on the other hand, resides on the local machine and is tied to the User ID who created it. The third type, System, can be used by anyone who has access to the machine, and is the type we’re going to create. Then we’ll use this DSN inside VFP.

Creating a System DSN To create the System DSN: 1. On your Windows machine, open up the ODBC Administrator. 2. Click System DSN tab, as shown in Figure 8.

Figure 8 . The System DSN tab of the Data Source Administrator allows you to create new DSNs for the box. 3. Click the Add button. 4. The Create New Data Source dialog appears as shown in Figure 9.


Figure 9 . Creating a new DSN using the MySQL ODBC driver. 5. Select the MySQL ODBC driver. 6. Click Finish. 7. The MySQL Connector/ODBC dialog appears, as shown in Figure 10.

102


Figure 10 . Selecting the MySQL ODBC driver opens a MySQL dialog for setting parms. 8. Enter the following settings: name: a unique name, such as 'mysqltest' description: enter a description for this driver if you like (it shows up in the ODBC Admin dialog's Configuration page) server: (leave blank – the driver will default to localhost, which makes sense since the MySQL server will be on the same machine as VFP) user: bob (this is the mysql user) password: secret (the password for bob, the mysql user)

If you’re going to try your first connection attempt with a MySQL server running on a Linux box, you’ll need to enter the name of the server in the “server” textbox. You can enter the name of the machine or the IP address. No entry for the database is necessary, since we’re just connecting to the MySQL server, but not opening a specific database. See Figure 11 for an example of what the dialog should look like.


Figure 11. The ODBC Connector dialog with appropriate values filled in. Don’t click the ‘OK’ button yet – we want to test the parameters first.

Testing the system DSN While you’re still in the ODBC dialog, it’s time to test the connection. Click the “Test” button. If it works, you should get a dialog indicating your good fortune, as shown in Figure 12.

Figure 12 . Success – you’ve connected to the MySQL server using the DSN.

104


Dealing with errors If you have not been quite so lucky as to have had a successful test immediately, here are a couple of troubleshooting ideas. The first thing that might happen is a general failure, as shown in Figure 13.

Figure 13 . A general failure message. The “SQL_ERROR” message isn’t very helpful, so close the error dialog, and then click the Diagnostics button in the ODBC dialog where you’ll see diagnostic messages like those shown in Figure 14.

Figure 14 . The diagnostics panel in the ODBC configuration dialog.


Upon stumbling into a message like this, you might want to do the equivalent of asking the user if the printer is plugged in – check to see if the MySQL server is running. Don’t laugh, it’s happened to this author once or twice. An alternative version of this message is Can't connect to MySQL server on '192.168.1.11' (10061)

With an IP address, this error likely means you can’t reach the machine 1.11 (the machine that the MySQL server is running on) from your Windows machine. This isn’t a MySQL problem as much as a network error, such as a firewall problem or the wrong IP address. Older versions of the ODBC configure dialog had a dialog that allowed you to try to ping the 1.11 machine. In the name of progress, that dialog has been replaced by the generic error dialog shown in Figure 13, so you’ll have to ping the box yourself. (Open a DOS box and type “ping localhost” or “ping 192.168.1.11” – and Ctrl-C to stop the ping process. Note that Windows stops pinging after the fourth iteration while *nix does not.) If you can’t ping the box, it’s likely an IP or network plumbing issue. If you can ping it but can’t reach it from the ODBC test dialog, the more likely culprit is a firewall. Let’s take a look at what some of the other typical error messages you might see in the diagnostics pane of the ODBC configuration dialog (all errors generate the same generic failure dialog in Figure 13.) Another problem you might run into is a message that says something like Unknown MySQL server host '192.168.1.777' (11001)

This error happens when the host identified in the Server text box on the Login tab of the ODBC driver dialog (Figure 11) is bad. You’ll need to do some sleuthing on your machine to find out why. In this example, you can see that the host, 192.168.1.777, is obviously incorrect (as none of the octets in an IP address can’t have a value > 255.) The next error you might encounter looks like this: Access denied for user 'roooot'@'localhost' (using password: YES)

This error means that the user credentials – either the username or the password (or both) – that were entered in the dialog in Figure 11 are bad. Again, you’ll need to do some investigating on your own. In this example, you can see that the username ‘root’ was misspelled. If you receive an error dialog that says something like Host '192.168.1.7' is not allowed to connect to this MySQL server

it means that you’re successfully connecting to the machine and finding the server, but the MySQL server isn’t allowing you in. Specifically, the user ‘bob’ isn’t allowed to access the MySQL server from the host you’re connecting from (in this example, the host being 192.168.1.7). You need to add a record for ‘bob’ that will allow him to access the MySQL server from the 1.7 machine. See the “Adding a work-a-day user” section in Chapter 5.

106


A VFP-specific DSN option Now that you’ve proved that the basic network plumbing is working, you need to make one setting change so the DSN works with VFP. Strictly speaking, you can wait until later, but if I don’t include it now, you (and I) will forget later, and then all sorts of arcane errors will start showing up. Click on the Advanced tab in the dialog, and check the “Don’t Optimize Column Width” check box, as shown in Figure 15.

Figure 15 . Setting the Don’t Optimize Column Width option in the ODBC driver dialog. MySQL, by default, returns the real width of a column, but this action isn’t looked upon kindly by Visual FoxPro. Specifically, you need the “Don’t Optimize” checkbox selected for memo and integer fields. Checking this box forces MySQL to return the defined width, much more to VFP’s liking. Specifically, two things can happen without the “Don’t Optimize” setting. First, with respect to views, if a field value gets truncated in a view query/requery, VFP will return error 1494 - View definition has been changed. Second, with respect to memo fields, if a MySQL query returns a character field of 255 characters or less, VFP will create a character field in the resultant cursor instead of a memo field. Then, if you perform any memospecific operations on this field, you’ll get error 350 - Field must be a memo field.


You may also want to check the “Pad CHAR field to full length” option on the “Flags 2” tab, which forces the MySQL server to fill out its varchar fields to the maximum length specified. If you don’t do this, some of your queries may return erratic results. Once you select the “Don’t Optimize” check box (and any others), click the OK button. Now it’s time to go to the next section where you use the DSN to connect to the MySQL server from within Visual FoxPro.

Use the DSN on the Windows machine to connect from VFP Now that the connection via the DSN is working, we’ll use the DSN to connect from within VFP. Open Visual FoxPro, and enter ? sqlconnect("mysqltest")

in the Command Window, where ‘mysqltest’ is the name of the System DSN you created in Figure 11. You should get a response back in the VFP desktop that simply says ‘1’, as shown in Figure 16.

Figure 16 . A successful SQLCONNECT command returns a value of ‘1’. If you have already created a connection, you will get a value other than 1. For example, if you execute SQLCONNECT three times, you should get return values of 1, 2, and 3. These represent three separate handles to the three connections you just created.

How to disconnect When you’re ready to close the connection, you issue the sqldisconnect() function, passing the value of the handle that was returned by the sqlconnect() function a moment earlier, like so: ? sqldisconnect(1)

Obviously, you’re not going to pass hard-coded values of connection handles in your applications. Instead, you’ll assign the return value from the sqlconnect() function to a variable or property, and then pass that variable/property to sqldisconnect() as the parameter: m.liHandle = sqlconnect("mysqltest") m.liSuccess = sqldisconnect(m.liHandle)

108


And m.liSuccess will either be a positive integer if the command worked, or negative if it failed.

Closing all connections You can also pass a parameter of zero to close all open connections: ? sqldisconnect(0)

Success, again, is indicated with a return value of 1. By the way, exiting Visual FoxPro also closes all open connections automatically, as you probably already guessed.

When to use (iHandle) and when to use (0) In some cases, it is safer to use the SQLDISCONNECT(0) option. If you instantiate an object, do some work, and then exit I would recommend using the global disconnect option (passing a parm of 0). Why? Well, Internet based systems aren’t like networked systems where you can create a connection and then create and release connection handles at will. The connection needs to be created and released as well as the handles in an Internet-based environment. For example, there have been instances using VFP 7 in Internet applications where a filebased PRG closed the connection via file handles, not using the SQLDISCONNECT(0) option. Unfortunately, the connections all piled up and became an operational issue. If your system is on a network, you can use the specific handle number, but if you’re accessing data over the Internet and creating a connection as a result of a web page hit, use the global disconnect option. With an Internet application, where you can’t be sure of what type of user load you’ll encounter, it always pays to be as frugal with connections as possible.

Connecting to MySQL from VFP with a connection string Now, we’ll build a manual connection string from the DSN created in the last section, and use that string to do the same connection to VFP. Along the way, we’ll learn the basics of issuing commands to the MySQL database server from within VFP.

Creating a connection string to connect from VFP The magical question has always been “How do you determine what the syntax of a connection string is?” You could make random guesses, of course, or, if you had a million monkeys and a million typewriters, you could have them do it for you. But the easy answer is to use VFP to grab the parameters from an existing connection that you already know works. Here’s an example. m.liHandle = sqlconnect("mysqltest") m.lcX = sqlgetprop(m.liHandle, "ConnectString") ? strtofile(m.lcX, "\teststring.txt")

The first line creates a connection, and assigns the connection handle to a variable that you’ll use in the next line. The second line grabs the connection string from the connection via the SQLGetProp function. It passes the handle of the connection you want the string for and the property you’re interested in (“ConnectString”) and stores the result – the connection string – to a variable.


Finally, in order to save yourself some grief, dump the contents of the variable to a text file using StrToFile or save the results of sqlgetprop to _cliptext, like so: _cliptext = sqlgetprop(m.liHandle, "ConnectString")

This way, you can open up the text file and copy the data, or simply grab it from your clipboard instead of relying on typing the values echoed to the screen, and possibly switching a ‘0’ (‘zero’) and an ‘O’ (‘oh’), or a ‘1’ (‘one’) and an ‘l’ (‘el’). (Other common mistakes are mixing up quotes and putting spaces in where they don’t belong and vice versa.) Doing so produces a text file with the following contents (given the parameters shown in Figure 11): DSN=mysqltest;UID=bob;PWD=secret

You’ll note that the name of the driver isn’t included. Since the goal of a connection string is to not need a DSN, we’ll replace the DSN identification with the explicit driver name. The words in between the braces come from the text you see in the Drivers tab of the ODBC Database Administrator in Figure 9. In this case, the syntax for the driver looks like this: DRIVER={MySQL ODBC 3.51 Driver}

A couple of notes about this string – those are braces, not parens, surrounding the name of the driver, and the spaces have to be exactly where they’re shown. If you installed a different version of the driver, say, version 3.52, the driver name portion of your connection string would be something like this: DRIVER={MySQL ODBC 3.52 Driver}

With all this, here’s what the ultimate result will look like. Note that this VFP command is using the “SQLStringConnect” function because you’re passing a string, not a named connection. If successful, the command will display a numeric value in the VFP desktop just like before. ? sqlstringconnect("DRIVER={MySQL ODBC 3.51 Driver};UID=bob;PWD=secret")

If you have additional pieces of information in your connection string, don’t worry about it yet – I’ll discuss those in the section “Additional connection string options” coming up shortly. Even this small connection string can be intimidating the first few times you look at it. As they say, “In order to eat an elephant, first cut it up into very small pieces.” Let’s cut this string up and look at the pieces.

Dissecting the parts of a connection string In order to help save on space so that as many of the examples fit on one line as possible, I’m not going to include the ‘=sqlstringconnect()’ part in the following discussion and examples. Everything I describe will go inside the function’s parens. A connection string is made up of key-value pairs, each separated by a semi-colon.

110


A key-value pair looks like SERVER=localhost

while multiple key-value pairs are strung together like so: SERVER=localhost;UID=bob;PWD=secret

The use of the semi-colon can be especially confusing to VFP developers since it is being used as a parameter separator, not as a line-continuation character as we are accustomed to. That means if you end up with a connection string that spans multiple lines, you’ll want to be doubly careful about where you put the key-value pairs and their semi-colons. There doesn’t have to be a semi-colon after the last key-value pair anymore than you’d include a comma after the last item in a list.

Additional connection string options Depending on how you set up your connection information (say, suppose you entered a description in the dialog in Figure 11), you might have already found additional strings in your connection string. The following contains an entry for the server – localhost in this case. "DRIVER={MySQL ODBC 3.51 Driver};SERVER=localhost;UID=bob;PWD=secret"

And this connection string has a string for the “Don’t optimize column width” setting in the Advanced tab of the ODBC dialog. "DRIVER={MySQL ODBC 3.51 Driver};OPTION=1;UID=bob;PWD=secret"

The sudden appearance of more and more parameters in the connection string can soon lead one to think that you’ll never learn all of the possible values, and that you better just stick to the DSN wizard. Not necessarily! Putting together a complete list is straightforward, once you learn the tricks. So far you’ve seen the key-value pairs for most of the controls in the Login tab of the ODBC dialog. To round out the list, here is the complete set of key-value pairs for the Login tab: DRIVER={MySQL ODBC 3.51 Driver} DESCRIPTION=A helpful phrase about the connection SERVER=localhost UID=bob PWD=secret DATABASE=test

The DATABASE parameter is useful if you’re setting up a connection to a known database. In some applications, you may be accessing more than one database, and will switch between them using the same connection, so you wouldn’t need to specify the database here. The Connect Options tab of the ODBC dialog, shown in Figure 17, also has associated key-value pairs that can be put into a connection string.


Figure 17 . The Connect Options tab of the ODBC dialog allows you to specify non- standard port and socket connection information. The PORT parameter explicitly identifies the port that you’re connecting to the MySQL server on. If you don’t include it, the connection assumes the default 3306. Some reviewers of this book have noted that they’ve had to use non-standard ports for their MySQL installation, like so: PORT=3307

The Socket parameter (shown in Figure 17) is only used if you are connecting to the MySQL server named “localhost.” The value here is the name of the Unix socket file or Windows named pipe. The Initial Statement is a command to execute when connecting to MySQL. Both of these are beyond the realm of this book. If you’ve already opened up the Advanced tab of the ODBC configuration dialog, you may be concerned about the two dozen or so choices in the various FlagN tabs. “Do I really need to learn and specify two dozen more parameters in my connection string?” you’re thinking. The answer is no. Instead, you assign a binary combination of flags that represent each option to the OPTION parameter, like so: OPTION=3

The 22 choices in the Advanced tab each have a numeric value assigned to them. Each value is a different power of two, so you can uniquely identify any combination of choices with a single value. For example, the “Don’t optimize column width” option has a value of 1,

112


the “Return matching rows” option has a value of 2, and the “Safe” option has a value of 131,072. If you wanted to create a connection string that included all three of these options, you’d add 1+2+131072 (which totals 131,075), and then include a key-value pair of OPTION=131075

in your connection string. The values for each choice in the Advanced tab are listed in section 23.1.9.4 “Connection Parameters” of the on-line help. Stringing a bunch of parameters together can lead to a rather long connection string, like so: ? sqlstringconnect("DRIVER={MySQL ODBC 3.51 Driver};SERVER=192.168.1.11;PORT=3307;OPTION=3;UID= bob;PWD=secret;database=test")

When you decide to break this string onto multiple lines, don’t forget the beginning and ending double quotes that delineate the entire connection string, and to take care of the semicolons that separate each parameter. The truly particular might do it something like this: ? sqlstringconnect("DRIVER={MySQL ODBC 3.51 Driver};" ; + "SERVER=192.168.1.11;" ; + "PORT=3307;" ; + "OPTION=3;" ; + "UID= bob;" ; + "PWD=secret;" ; + "database=test")

Of course, your mileage may vary.

Dealing with errors programmatically Your first hint that something is amiss is that you don’t get an immediate response; typically, a successful connection returns a value before you can blink. A connection attempt that fails can take 5 to 10 seconds before the dreaded ‘-1’ appears. If you’ve mistyped the connection string, like so: m.lcXN = "{DRIVER=MySQL ODBC 3.51 Driver};SERVER=127.0.0.1;UID= bob;PWD=secret;"

(the error is that the opening braces is before the word “DRIVER” instead of before the word “MySQL,” which is a common problem), you’ll get a dialog box asking you where the driver is, as shown in Figure 18.


Figure 18 . The Select Data Source dialog appears if you mistype a driver name. Other times, errors can be more obscure. Error information can be captured via the AERROR function, like so: ? sqlstringconnect("DRIVER={MySQL ODBC 3.51 Driver};" ; + "SERVER=192.999.999.999;" ; + "UID= bob;" ; + "PWD=secret;) dimension abc[1] ? aerror(abc)

(You’ll notice that the IP address for the server is invalid, thus generating an error.) Open up your Locals window (found under the Tools menu if the Debugger is in the FoxPro frame or under the VFP Debugger’s Window menu if you’re running the debugger in its own frame) as shown in Figure 19

Figure 19 . The Locals window allows you to easily drill down into an AERROR array.

114


and spelunk through the results. Unfortunately, the errors returned by the driver can be very misleading. In this example, I specified a bad server name, which means the driver wasn’t going to be able to find the server. The error returned, “Could not find driver...”, doesn’t make sense – the error should have been something like “Could not find server.” If you don’t want to go through the trouble of using the Locals window, you can simply issue a display memory like abc

command and view the results on the VFP desktop. And now back to our regularly scheduled program.

DSNs or connection strings? As with everything in Visual FoxPro, there are a dozen ways to perform a task. Sometimes even thirteen ways. And if you ask 13 developers which way is the best, each will argue vociferously that their way is the best. So how do you choose? Specifically, what are the pros and cons of DSNs and connection strings? The primary advantage of a DSN is that it is stored in the Windows Registry, which keeps it reasonably secure from prying eyes and others who would inadvertently or malevolently cause problems. (You could even prevent access to it altogether using Windows’ Group Policies, a topic worth investigating but beyond the scope of this book.) Furthermore, it is quite possible to create a routine to check for the DSN upon application startup – see Chapter 13 of MegaFox, “How can I be sure users have the correct settings,” for details. You could even reinstall the DSN programmatically during startup, exporting the ODBC settings from the Windows Registry as a .REG file, and then execute it on other machines using ShellExecute(), if you needed to. This way users can update settings automatically on application startup and DBAs can retain control over connection information. The main disadvantage of DSNs, especially System DSNs, is that they are available to any application running on your system. A fairly smart user then could fire up a copy of, say, Microsoft Excel and connect to your data using the DSN which has the username/password stored in it. The advantage of connection strings is that they are infinitely flexible, as they are built at run-time. It’s not a difficult task to store connection string parameters in the system’s meta data (encrypted, please, since user logins and passwords are part of the package) and retrieve them as necessary. Some developers may feel more comfortable manipulating system meta data instead of having to get into the Windows Registry. I personally prefer connection strings to DSNs, but that’s just a personal preference. I don’t have a passionate view of one over the other, and suggest that you try on both for size, and see which fits to your liking.

Initial tests with Visual FoxPro and MySQL Now that you have a connection between VFP and MySQL set up and working, it’s time to use it. You know, create databases and insert records into tables, that sort of fun stuff. Fire up VFP, open the Command Window, create a connection, and assign the handle to a variable (I like using a short variable name since I’m going to be using it over and over again): m.liH = sqlstringconnect("DRIVER={MySQL ODBC 3.51 Driver};" ;


+ "SERVER=192.168.xxx.xxx;" ; + "UID= bob;" ; + "PWD=secret;)

Creating a database Now that you have a handle, let’s work with MySQL. First, we need a database to work with. Let’s create one. * Create a database to experiment with ? SQLEXEC(m.liH, "CREATE DATABASE test_cust")

In this code segment, we’re doing a couple of things. Just like you can issue a SQL statement in the VFP Command Window that works against native VFP tables, you can send SQL statements to a back-end database via the SQLEXEC command. By the way, SQLEXEC (and SQLCONNECT, SQLSTRINGCONNECT, and SQLDISCONNECT) is an example of one of three methods to work with MySQL data from within VFP - SQL Pass Through (SPT). The other two methods, Remote Views and CursorAdaptors, is discussed, along with a more thorough discussion of SPT, in Chapter 15. It takes, as you can see, two parameters. (Actually, it can take more, but we just care about the first two for now.) First, we have to identify which connection we’re working with, which will identify what database server, and, possibly, which database. That’s taken care of by the m.liH variable, which contains the connection handle. Second, we need to send the actual SQL command we wish to execute. For short commands (a couple hundred characters or less), you can just enclose the entire command in quotes as shown below. Longer commands require a bit of sleight-of-hand that I’ll cover later in this book. Executing the command as shown with the “?” will display the return value on the VFP desktop. If successful, the return value will be ‘1’; if unsuccessful (say, due to a syntax error), the return value will be a ‘-1’, like so: * Issue a command with incorrect syntax ? SQLEXEC(m.liH, "CREATE DBASE test_cust") -1 * Now issue a command with correct syntax ? SQLEXEC(m.liH, "CREATE DATABASE test_cust") 1

If you’ve been following along precisely, you probably got a -1 after the correct CREATE command – this is because you already created the database correctly, and you can’t overwrite an existing database, so the second attempt failed. Also, if the value of SQLGetProp(m.liH, “DispWarnings”) = .t., you’ll get an error that looks the dialog shown in Figure 20.

116


Figure 20 . If “Display Warnings” is turned on. You’ll definitely want to turn that OFF in your applications by issuing a SQLSetProp(m.liH, “DispWarnings”, .f.) command. You’ll also find that the incorrect CREATE command moved you off the database that the successful CREATE command had created, so you’ll want to select it again, much like you select a work area in Visual FoxPro: ? SQLEXEC(m.liH, "USE test_cust")

Again, if you mistype the command, or if test_cust doesn’t exist, the SQLEXEC command will return a -1. It’s a good time to mention that if you’re one of those folks like to type VFP four-letter abbreviations, now is the time to get over it. SPT will not accept “crea data test_cust” like VFP does. You’ll also want to get over the habit of using the command “USE” to open a table. MySQL’s use of “USE” is comparable to VFP’s SET DATABASE TO command. Since there are no objects like VFP’s work areas in MySQL, there isn’t a comparable MySQL command to “USE” either.

Where is the data created? A directory named “test_cust” is created under the default data location specified during installation. I’ll discuss these values in more detail in Chapter 7 on Configuration. In Windows, the default is the data directory under where you installed MySQL (c:\Program Files\MySQL\data) unless you changed either the location of the MySQL install or the default data directory. In Linux, the default data directory is /var/lib/mysql. Details on MySQL data are covered in Chapter 9, Under the Hood. Inside the directory, you’ll see a file named db.opt, which contains the settings (such as default character set and collation sequence) for the database you created.

Working with tables in our new database Now that a new database, named test_cust, has been created, the next logical step is to create a table in that database, and then add records to that table: * ? * ? *

Make sure we're working with the right database SQLEXEC(m.liH, "USE test_cust") Create a table in the test database SQLEXEC(m.liH,"CREATE TABLE cust (cName CHAR(25))") Insert some rows into the table


? SQLEXEC(m.liH,"INSERT INTO cust (cName) VALUES ('Data Schmata 1')") ? SQLEXEC(m.liH,"INSERT INTO cust (cName) VALUES ('Data Schmata 2')") ? SQLEXEC(m.liH,"INSERT INTO cust (cName) VALUES ('Data Schmata 3')")

Our last task at the current time (remember, all we’re doing right now is proving that the connection does indeed work) is to prove that the data was actually put in the table. We can run a SQL SELECT command that returns the contents, like so: * prove the data was put in - should create a cursor named csrWhatIPutIn, * with the records that were just added ? SQLEXEC(m.liH, "SELECT * from cust", "csrWhatIPutIn")

You’ll note a third parameter in the SQLEXEC function this time. This is the name of the result set that will be created by the SQL SELECT command. If you perform a SQLEXEC command that returns a result set (i.e. a SELECT as opposed to an UPDATE), VFP will by default name the result set “sqlresult” unless you explicitly provide the name of the result set. You always want to name the result set yourself, because VFP will overwrite an existing “sqlresult” cursor without warning. Even if you’re going to throw the cursor away a moment later, take the time to name it explicitly – perhaps, like “csrJunk”. A common mistake (at least for me!) is to forget to enclose the cursor name in quotes – if you don’t, VFP will try to create a cursor with the name represented by the contents of the variable “csrWhatIPutIn,” which more than likely doesn’t exist. You can also run a SQL SELECT command that returns just the number of rows in the table, like so: * * * ?

now run a command that creates a cursor named csrHowMany, with a field named iHowMany, that contains the value 3 (or however many times you executed the INSERT command) SQLEXEC(m.liH, "SELECT count(cName) as iHowMany from cust", ; "csrHowMany")

The SQL statement in this command will generate a record, one field cursor named “csrHowMany.” The value in this field should be the number of records you inserted (3, unless you experimented and added more or less.) You’ll also note that the command didn’t fit onto one line, and so in this example, I used the VFP line continuation character to wrap to a second line. Because SQL commands can get quite lengthy and keeping track of semicolons and delimiters can get tedious, a common workaround is to store the SQL command in a variable. You can use the TEXT...ENDTEXT command in VFP, like so: TEXT TO m.lcSQL NOSHOW PRETEXT 3 SELECT count(cName) as iHowMany FROM cust ENDTEXT ? SQLEXEC(m.liH, m.lcSql, "csrHowMany")

You can execute all DML (Data Manipulation Language) SQL commands, including SELECT, INSERT, UPDATE, and DELETE, as well as all DDL (Data Definition Language) SQL commands (CREATE DATABASE, CREATE TABLE, ALTER TABLE, etc.)

118


Finally, I want to mention that these examples are a classic example of melding VFP with MySQL – using SQLEXEC to get data from the database and dropping it into a VFP cursor that can be manipulated with VFP’s rich language constructs. Now that we’re convinced that we can connect to MySQL and work with it a bit, it’s time to close up shop. Use the SQLDISCONNECT command, like so: * Close the connection ? SQLDISCONNECT(m.liH) 1

If the disconnect is successful, you’ll get a return value of ‘1’. An unsuccessful disconnect (say, for example, you used the wrong handle) will generate a negative value. The Developer Download files for this chapter, available at www.hentzenwerke.com, include FirstConnectionTest.PRG. Note that you’ll need to change the connection string parameters to reflect the credentials on your machine.

Listing 1. FirstConnectionTest.PRG. * Set up the connection string m.lcXN="DRIVER={MySQL ODBC 3.51 Driver};SERVER=127.0.0.1;UID=bob;PWD=secret;" * Connect to it and get a handle m.liH=SQLSTRINGCONNECT(m.lcXN) * Create a database to experiment with ? SQLEXEC(m.liH,"CREATE DATABASE test_cust") * Create a table in the test database ? SQLEXEC(m.liH,"CREATE TABLE cust (cField CHAR(25))") * Insert some rows into the table ? SQLEXEC(m.liH,"INSERT INTO cust (cName) VALUES ('Data Schmata 1')") ? SQLEXEC(m.liH,"INSERT INTO cust (cName) VALUES ('Data Schmata 2')") ? SQLEXEC(m.liH,"INSERT INTO cust (cName) VALUES ('Data Schmata 3')") * prove the data was put in - should create a cursor named csrWhatIPutIn, * with the records that were just added ? SQLEXEC(m.liH, "SELECT * from cust", "csrWhatIPutIn") * now run a command that creates a cursor a cursor named csrHowMany, * with a field named iHowMany, that contains the value 3 (or however * many times you executed the INSERT command) ? SQLEXEC(m.liH, "SELECT count(cName) as iHowMany from cust", ; "csrHowMany") * Close the connection ? SQLDISCONNECT(m.liH)

Conclusion/Summary Now you’ve proven that MySQL is running and you can connect to it from Visual FoxPro. For many developers, reaching this point is the Holy Grail, and indeed, it can be frustrating to assemble all the steps just to get this far. But we’re just beginning the adventure. In the next couple of chapters, we direct our attention to working with MySQL directly. Once we have those tools in our arsenal, we’ll start building a Visual FoxPro application with MySQL. Updates and corrections to this chapter can be found on Hentzenwerke’s Web site, www.hentzenwerke.com. Click “Catalog” and navigate to the page for this book.

Chapter 7: Configuring MySQL 119

Chapter 7 Configuring MySQL MySQL’s operation is controlled by a configuration file. You can use a graphical frontend, called the MySQL Administrator, to change this file, or you can edit this file directly. In this chapter, I’ll first show you how to install and use the Administrator. Once you’re comfortable with this tool, I’ll cover the essential configuration items you’ll want to look at and possibly tweak yourself, and then briefly cover the rest of the settings. Finally, I’ll discuss how to edit the configuration file directly.

The MySQL configuration file is a plain text file that is referenced during the startup of the MySQL server as well as MySQL client applications. The documentation inside the file is outstanding, and I urge you to read through it even if you’re not planning on ever editing it directly yourself. But first, you need to know where to find it.

The MySQL configuration file The configuration files used by MySQL are structured in a very flexible manner. Unfortunately, in this situation, ‘flexible’ is marketing-speak for ‘confusing’, because different platforms use different, but also overlapping, files.

my.ini in Windows On Windows, MySQL server looks for a file named “my.ini.” While the my.ini file is typically in the same directory as the mysql executable, an argument is passed to the MySQL service command that explicitly identifies where this file is. On Windows, the syntax looks like this: "C:\mysql\bin\mysqld-nt" --defaults-file="C:\mysql\my.ini" MySQL

You can find out what my.ini file is being used in your Windows installation via the Services applet. Click on the Services applet in the Administrative Tools window in Control Panel to open the Services list, as shown in Figure 1.

120


Figure 1. Locating the MySQL service in the Services applet. Right click on the MySQL service and select Properties to bring forward the Properties dialog as shown in Figure 2.


Figure 2 . The location of the my.ini configuration file is in the executable path. You’ll see the configuration file being used by the current MySQL service displayed in the read-only text box under “Path to executable:” – the text box is read-only because it is normally modified via manual MySQL commands or through the GUI tool.

my.cnf in Linux On Linux, MySQL looks for a configuration file named “my.cnf” located in the /etc directory: /etc/my.cnf

This file controls global options. Because you can have multiple MySQL databases running on a single physical machine, you might think it would be convenient to be able to specify specific configuration settings, and you can. You can specify server-specific options by placing a copy of this file in the data directory for the server, like so: /var/lib/mysql/my.cnf

122


and modifying the file accordingly. Even further fine-tuning is available on a user-by-user case by placing a copy of this file in the user’s home directory, like so: /home//my.cnf

and setting user-specific options. To further complicate matters, you can also use my.cnf files on Windows, but at this point, we’re venturing into complexities unnecessary for our current purposes. Let’s move along and get into the MySQL Administrator, a GUI tool used to make changes to the configuration file graphically.

Downloading the MySQL Administrator The MySQL Administrator is available from the same locations that the MySQL server itself is from – see Figure 3. Go to http:\\dev.mysql.com, click on the Downloads menu (far left), and navigate down the page to “MySQL Tools.” Click on “MySQL Administrator,” which brings you to the page shown in Figure 3.

Figure 3 . Locating the download link for the MySQL Administrator.


Clicking the link takes you to a list of files for various operating systems, as shown in Figure 4.

Figure 4 . Identifying the specific MySQL Administrator downloads. Clicking the download link in the Windows section will eventually download a file named mysql-administrator-1.1.9-win.msi

(or later) to your hard disk; clicking the download link in the SUSE Linux 9.3 section will download a file named mysql-administrator-1.1.6-1.suse93.i586.rpm

in a similar manner.

124


Installing the Administrator On Windows Right-click on the .msi file you just downloaded and select the “Install” option from the context menu. Doing so displays the starting dialog as shown in Figure 5.

Figure 5 . Starting up the Administrator Wizard in Windows. Clicking the Next button brings you to the license screen, shown in Figure 6.

Figure 6 . The Administrator license agreement.


Clicking the Next button takes you to the Destination Folder dialog. At this point, the default is usually under C:\Program Files – if you changed the default installation directory of MySQL to something like C:\mysql, then I suggest you change the installation directory of the administrator also so it is still under the main MySQL directory, like so: c:\mysql\admin1.1

Figure 7 shows the Administrator under Program Files.

Figure 7 . Pointing to the installation directory for the Administrator. Choose the Custom install if you want to control which components are installed. There are only two; one of which is optional, so it’s not that big a deal. Still, I usually choose Custom, as shown in Figure 8, so I can see what’s included.

126


Figure 8 . Selecting the type of installation. Figure 9 shows you the two components – the Administrator itself and a System Tray Monitor, which I recommend you include since it takes virtually no space and is darn tootin’ handy.

Figure 9 . Selecting the Administrator components.


Now it’s time to do the install, as shown in Figure 10.

Figure 10 . Executing the installation. The thermo bar will scroll across for a minute or so, as seen in Figure 11.

Figure 11. Waiting for the installation to complete.

128


And then you’re done – as shown in Figure 12.

Figure 12 . Success! If you’re not installing on Linux, move on to the section titled “Running the Administrator.”

On Linux Assuming you put the RPM file in the ‘zips’ directory under your home directory, you can simply enter rpm -i /home//zips/mysql-a

and hit the Tab key to take advantage of the command completion feature in Linux, and then press Enter to install manually. You could also use YaST (Yet another Setup Tool). Double-click on the RPM file in Konqueror and select the “Install package with YAST” button as shown in Figure 13; or select the file in Konqueror, right-click, select Actions, and then Install with YaST.


Figure 13 . Installing MySQL Administrator via YaST. You should then get the ‘root user’ needed dialog, as seen in Figure 14,

Figure 14 . Installation requires root permission. The blank YaST screen appears. Enter ‘mysql’ in the search text box and you’ll get a bunch of packages, including the one you highlighted. Note that other packages are part of the SuSE install, and thus are probably older than what you installed in Chapter 4. Select the Administrator as shown in Figure 15, and then click the Accept button in the lower right.

130


Figure 15 . Identifying the MySQL Administrator package in YaST. You’ll get the mysql-administrator package installation thermometer bar in the bottom of the screen as shown in Figure 16.

Figure 16 . Executing the package installation. Once finished, close the dialog, and you’re all set to run the Administrator.


Running the Administrator In my opinion, the best way to run the MySQL Administrator in Windows is through the MySQL System Tray Monitor. On Windows, the System Tray Monitor should have been installed along with the rest of the MySQL Administrator process. If it wasn’t (perhaps you forgot to select the option or Windows hiccupped during the installation), you can install the monitor into the System Tray via the Start | MySQL | MySQL System Tray Monitor menu option. Doing so will ‘run’ the System Tray Monitor – which places an icon that looks like a life preserver with a blue bar on its right side into the System Tray, as shown in Figure 17 (immediately to the left of the time).

Figure 17 . The MySQL Monitor context menu. Right clicking on the MySQL System Tray Monitor icon displays the context menu also shown in Figure 17. Launch the MySQL Administrator via its menu option. (Note that if you run the System Tray Monitor before installing an option, such as the Query Browser, that option will appear disabled until you close the monitor and open it again.) In Linux, you can either select the Applications, MySQL Administrator menu option from the Gecko start menu button, or execute /usr/bin/mysql-administrator from a command prompt. In both cases, you’ll be greeted by the login screen displayed in Figure 18.

132


Figure 18 . The MySQL Administrator login screen. On Windows, you may be asked to poke through your firewall. Figure 19 shows ZoneAlarm prompting said action.

Figure 19 . Giving the Administrator permission to the ZoneAlarm firewall. Once you open the MySQL Administrator, you’ll be greeted with a window like that in Figure 27. We’ll come back to this dialog in a minute. First, let’s go over the various options in the login dialog from Figure 18.


Advanced login options Some of the default options for the Login dialog were shown in Figure 18: Server Host of ‘localhost’ and Port of ‘3306’. Obviously the username and password will be different according to your own installation and how you’ve created user accounts. First, you can choose to connect to a MySQL server running on a different machine by entering that machine’s name or IP address in the text box. For example, if the machine that MySQL is running on is on the same network and is named ‘herman’, you would just enter Server Host: herman

If you wanted, you could use the IP address of the machine, like so: Server Host: 192.168.1.11

While MySQL is configured by default to allow connections via port 3306 (that’s the connections to the server, not your client), some installations may change that port number. If you need to connect to a MySQL server that listens on a non-standard port, enter that port into the Port text box: Port: 9906

You should create a profile called a “Stored Connection” for your server. (If you routinely connect to more than one MySQL server, you can create a connection for each server.) In order to create a stored connection, click on the ellipses button to the right of the Stored Connection combo box. You’ll get the “Connections” dialog shown in Figure 20.

134


Figure 20 . Creating a profile for a connection. Click on “Add new Connection” to make the text boxes in the Connection Parameters tab editable, as shown in Figure 21.


Figure 21. Creating the connection for a new profile. Enter a text string for what you want to call the connection (this string will show up in the Stored Connections combo box shown in Figure 18.) While some folks tend toward using whimsical names for their connections (“Happy,” “Sleepy,” “Grumpy,” etc.), or worse, generic names (“Connection1,” “Test2,” “Database”), I find I can’t remember what’s on the other end of a name like this. Instead, I’ll try to provide some specific details about where I’m connecting, such as the name of the box, the IP address, and/or the OS on the box. If I was flipping between multiple versions of MySQL, I’d probably include that info too. Don’t bother with the Password if you’re creating a connection of Type “MySQL” (see the combo box three rows below the Connection text box), as by default it’s not stored with the connection and you’ll have to type it in when you select the Stored Connection in the Login dialog later. If you want to have the password stored as part of the connection, you can select the “Store Passwords” in the General Options node as shown in Figure 22.

136


Figure 22 . General options for the profile. Note that you can choose to have your password stored in one of three methods. Once you’re done dealing with the password, enter the attributes of the MySQL server that this connection is going to connect to (back in Figure 21.) The Hostname of the server can be ‘localhost’, a machine name, or an IP address. The Port is typically 3306, but might be different depending on special needs. Keep the “Type” set to MySQL. Since you’re creating a connection for the MySQL Administrator, you won’t enter a Schema (the name of a database), but if you were creating a connection for use with the Query Browser, you could enter one if you wanted. Finally, add notes if desired. The result is shown in Figure 23.


Figure 23 . A completed connection, before it is saved. After clicking the “Apply” button, the name of the connection will be displayed under the Connections node in the tree view control, as shown in Figure 24.

138


Figure 24 . A saved connection for a profile. Click the Close button in the lower right corner, and the Login dialog with the newly created connection selected will be displayed, ready for you to log in to the server. See Figure 25.


Figure 25 . Selecting a profile. One final note: If you installed the Query Browser (see Chapter 8), the “Query Browser” menu option in the MySQL Administrator will be enabled. When you click on the “Query Browser” menu option in the MySQL System Tray Monitor, you’ll get one more control in the Login dialog, as shown in Figure 26.

Figure 26 . The MySQL Login dialog displayed from the Query Browser.

140


The Default Schema text box allows you to specify which database you want to start working with in the Query Browser. Details are in Chapter 8, “The Interactive Use of MySQL.” It will be filled in automatically if you enter a Schema when creating a connection as described above. Even though this text box doesn’t show up when you select the MySQL Administrator menu option, I wanted to point out this one subtle difference now, in case you accidentally chose the wrong menu option in the MySQL System Tray Monitor, and couldn’t figure out why that extra text box was showing up. Click OK to connect. Now on to the Administrator.

The Administrator The MySQL Administrator has 11 sections, grouped into nodes, and displayed in the left-side pane of the main window as shown in Figure 27. If you select “Configure Instance” in the MySQL System Tray Monitor context menu (Figure 17), you’ll get an abbreviated version of the Administrator, with only the Service Control, Startup Variables, and Server Logs nodes in the left-side pane. Additionally, if you connect to a remote server, several of the nodes will be disabled: Service Control, Startup Variables, and Server Logs.

Figure 27 . The main Administrator screen. As you can see from the various nodes in the list in the left panel, the MySQL Administrator gives you the ability to manage all aspects of the MySQL server from a convenient graphical user interface. In this chapter, we will be primarily concerned with configuring MySQL so we can start using it; administration will come later after we’ve developed a need for more than the ability to start and stop the server.


The Service Control node Clicking on the second node, Service Control, opens the Start/Stop Service tab, as shown in Figure 28.

Figure 28 . The Service Control node allows you to start and stop the server without having to mess with the operating system mechanisms. From here, you can start and stop the MySQL service via the “Stop Service” button (which changes to “Start Service” after you stop the service). When you start or stop the service, logging messages display in the Log Messages text box. This feature is particularly handy in that you have to stop and start the server for changes made to the configuration, both in the MySQL Administrator as well as manually. The Configure Service tab in the Service Control node, shown in Figure 29, allows you to set various parameters regarding the service.

142


Figure 29 . The Configure Service tab of the Service Control node allows you to specify aspects of the service startup. Before I get started on the contents of the form, note the scroll bar on the right side – once you make changes, you may need to scroll the form down to find the “Apply Changes” and “Discard Changes” buttons – they’re not visible on the form as it’s shown in Figure 29. (Depending on your screen resolution, you may be able to see the entire form.) The buttons are showing in Figure 30 as well as the last few options. You could also resize the form via the resizer in the lower right corner of the form in order to see more. By default, the MySQL service is automatically started; thus, the Launch MySQL server automatically checkbox will be selected. You can turn that flag off, as well as change the name of the service as it displays in the Properties dialog of the Services applet (shown way back in Figure 2). This can be handy if you install multiple versions of MySQL and need to keep them straight – “MySQL 4.0,” “MySQL 4.1,” and “MySQL 5.0” are easier to work with than “MySQL,” “MySQL1,” and “MySQL2”. You can also use this tab to point to a different configuration file. Just type its name and path in the text box. If the file doesn’t already exist, the string you type in will be displayed in red, as shown in Figure 30. Once you save your changes, the startup string for the service (you can see it in the Properties dialog in Figure 2) will reflect your new configuration file.


Figure 30 . Invalid values entered in text boxes are displayed in red. The “Launch MySQL server automatically” and “Config filename” settings are both stored in the Services applet in Windows and modify the scripts contained in /etc/init.d on Linux.

The Startup Variables node The next thing you will typically want to do is point to your database. These options are found in the Startup Variables node, as shown in Figure 31.

144


Figure 31. Pointing to the database in the configuration file. You’ll see that this node has lots and lots of tabs. There are two things that aren’t quite as obvious as this abundance of tabs. The first, which is a completely non-standard user interface mechanism, is the little picture with a red X to the left of various options. For example, there’s one to the left of the disabled “Temp directory” in the “Directories” box in Figure 31. This is a toggle that acts much like a checkbox, interestingly enough. As shown in Figure 31, the Temp directory field, although not grayed out, can’t be typed in, which means you can’t enter a temporary directory path. Click on the icon with the red X and the red X disappears, so it looks like the icons for “Base directory” and “Data directory.” Once you do so, you can enter a path in the text box. Doing so, as making other changes does as well, enables the “Apply changes” and “Discard changes” buttons in the lower right corner. As mentioned before, another thing that isn’t immediately obvious is the dialog is a variable height form. You can drag the lower right corner of the dialog to make the form taller, and thus be able to see more controls without scrolling. For example, in Figure 31, I stretched the form down so that the scroll bar disappeared – I wanted the “General” box at the bottom of the tab to display. By default, that box is out of sight. Also note that the name of the file that stores this information is displayed in the bottom of the form – in Figure 31 it says “C:\mysql\my.ini.” Yours may be “C:\Program Files\MySQL\MySQL 5.0\my.ini.”

General Parameters Depending on how you installed MySQL to begin with, your settings for “Base directory,” “Data directory,” and “Default storage” may differ from what is shown in Figure 31.


Furthermore, once you finish your install, you may wish that you’d made different choices. The General Parameters tab is where you rectify those decisions. The “Default storage” combo box has four choices: MyISAM, InnoDB, BDB (Berkeley Database), and Heap. This setting will control the type of database or table created if you don’t explicitly identify the table type in your SQL CREATE statement. (For details about the various storage types, see Chapter 14, “Storage Engines and Table Types,” in the on-line help.) As a result, it’s possible to create some tables in a database with one type and other tables in the same database of another type. For example, the default database type is InnoDB. You can open an InnoDB database and then manually add another table to it, explicitly specifying MyISAM as the type. Some readers prefer to mix and match MyISAM and InnoDB tables in the same database, using each storage engine for specific purposes. MyISAM tables are significantly faster, and thus are a good choice for write-once, read-many tables like error logs, audit trails, and query-intensive data warehouse files. InnoDB tables, on the other hand, since they support transactions, albeit at a cost of some performance, are good for the workhorse tables in the database. The “Base directory” identifies where the executable is. As the label on the dialog says, other paths (meaning relative paths) are resolved relative to the base directory. You’ll probably want to leave this alone. The “Data directory,” on the other hand, is potentially of much interest, particularly if you missed the step during installation where you could specify where you wanted your data to live. It contains three different things. First, the root data directory contains the location for the log and error files, prefixed with the name of the machine, so if your machine name was “frodo,”, the MySQL log file would be named “frodo.log” and the MySQL error file would be called “frodo.err.” The root also contains a “pid” (Process ID) file of the current MySQL instance, also named with the machine name, such as “frodo.pid.” If you shut down the instance, the PID file goes away. Second, there is a subdirectory named ‘mysql’. This directory contains the meta-data for MySQL – tables that identify the hosts, databases, table schemas, privileges, and so on. Even if you use a non-MyISAM database type, these tables are used by MySQL. Finally, there are additional subdirectories (on the same level as the ‘mysql’ subdirectory) – one for each MyISAM MySQL database identified in the meta-data. If you change the “Data directory” value, note that you’ll need to move the contents of the original directory to the new location, or else the server won’t be able to find the meta-data or the databases the meta-data references.

MyISAM parameters The MyISAM parameters tab of the Startup Variables node doesn’t contain any options that you’ll typically want to modify as you’re getting started.

InnoDB parameters The InnoDB parameters tab of the Startup Variables node is another story, though, if you’re using the InnoDB storage engine. Figure 32 shows part of the tab, as it’s another one of those “scroll down” jobbies.

146


Figure 32 . The InnoDB tab of the Startup Variables node. First, you’ll want to make sure the “Activate InnoDB” checkbox is selected. This setting correlates to the $skip-innodb

setting in the configuration file. If this setting (in the file) has a leading ‘#’, it means that it is commented out, and thus MySQL will NOT skip InnoDB – in other words, InnoDB will be used. If it’s not commented out, like so: skip-innodb

the InnoDB storage engine will be activated. Doncha just love double negatives? (Or maybe that should be “Doncha not hate double negatives?”) Next, you can change the location for the InnoDB databases if you don’t want them in the default data directory. Do this by scrolling down to the “Data directory” text box in the “Datafiles” box and entering the path for your InnoDB data files. By default, all of the tables in InnoDB databases are contained in a single file.

Other parameters Depending on your application, you may need to tweak the Max Connections parameter in the Networking tab. Other than that, we’re not going to bother with the parameters in any of the other tabs, such as Performance, Log files, Replication, and so on. You may want to scan through the controls in each tab, just to see if any light a particular fire for your particular scenario, but I’d argue that most readers will want to get on to the rest of the Administrator.


The User Administration node The User Administration node allows you to add, delete, and edit users and their privileges without having to manually work with the GRANT command as we did in the installation chapters. Selecting the User Administration node will display all of the MySQL users in the Users Accounts list box below the list of nodes. Selecting a user in the Users Accounts list box will then display that user’s account information in the User Information tab, as shown in Figure 33.

Figure 33 . The User Administration tab allows you to visually manage users and their permissions. Once you add a user, you can see which databases the user has access to, and their specific rights for each database. Click on the Schema Privileges tab, and then select a specific scheme in the list box on the left. Figure 34 shows that the user ‘bob’ has SELECT, INSERT, and UPDATE privileges to the TESTISAM database.

148


Figure 34 . Visually setting permissions for user ‘bob’. You can fine-tune the rights for each user, giving each person specific abilities on a database by database basis. If you haven’t already, you’ll want to create a user for your day-to-day use. While Windows has always been lax about encouraging you to run as a non-Admin user, the Open Source software world isn’t quite so forgiving. Logging on a Microsoft SQL Server as ‘admin’ with an ‘sa’ password wouldn’t even raise an eyebrow in many installations, but logging in as ‘root’ on a MySQL server will generate reactions ranging from mild disapproval to outright ridicule from the surrounding folks. So set that work-a-day user up now!

Other nodes The other nodes in the list are used during various system maintenance tasks. While we don’t need to discuss them now, I suggest you take a moment or three to click on each node and examine the functions in each of them. We’ll revisit them as appropriate later in the book.

Editing the my.ini/my.cnf file directly The MySQL configuration file carries on the tradition of excellent commenting in the file; indeed, after looking through the file, you may conclude that you don’t need to read anything here about how to change the settings (and you’d be mostly right.) When you make a change to the configuration file in the Administrator, the previous version is automatically backed up, so you can easily roll back to a previous version. If you spend some time goofing around with various settings, don’t be surprised to find a dozen or more files, looking like this: my.bak1 my.bak2


my.bak3 my.bak4 my.bak5 my.bak6 my.bak7 my.ini

in the MySQL root directory. That said, if you want to try modifying the configuration file by hand, don’t forget to make a backup, and remember that you’ll need to stop and start the server in order for your new settings to take effect!

Specific settings in the MySQL configuration file The easiest way to learn to directly modify the file is to make the desired changes in the MySQL Administrator, and then open up the file and see what has changed. Here are a couple of items in the configuration file that match the settings I discussed earlier in this chapter. First, look for the section that starts with these lines: # SERVER SECTION # ----------------------------------------------------------------

Underneath, you’ll see a couple of path statements: #Path to installation directory.... basedir="C:/mysql/" #Path to the database root datadir=e:/wsdb/mysqlisam/

These match to the values in Figure 31. A little below those lines is the specification for the default storage engine: # The default storage engine that will be used when creating new tables default-storage-engine=innodb

The setting for MyISAM tables looks like this: # default-storage-engine=myisam

Later on in the configuration file, there are sections for MyISAM and InnoDB specific options, identified by the headings: #*** MyISAM Specific options

and #*** INNODB Specific options ***

It’s educational to manually change settings in the configuration file, and then open up the MySQL Administrator to see how those manual changes are reflected in the GUI.

150


Conclusion/Summary The MySQL configuration is one of those things that you’ll typically set up once and then leave alone until a special need comes along. The MySQL Administrator, as a result, is a handy tool for modifying the configuration file without having to remember how the plumbing under the hood is constructed. Now that MySQL is running and configured, it’s time to start working with data for real. In the next chapter, we’ll explore how to use the MySQL Query Browser to work with MySQL data interactively. Updates and corrections to this chapter can be found on Hentzenwerke’s Web site, www.hentzenwerke.com. Click “Catalog” and navigate to the page for this book.

Chapter 8: The Interactive Use of MySQL 151

Chapter 8 The Interactive Use of MySQL Once the database server is up and running, and you have proven that you can connect to it from Visual FoxPro, what next? You’ll need to get comfortable using MySQL in an interactive way – creating databases and tables, and then adding, editing, and deleting data from those structures. In this chapter, I’ll discuss using the MySQL Query Browser to work with MySQL in an interactive session. After covering installation, I’ll tour the various parts of the interface, and then show how to use it to perform data definition (creating databases and tables) as well as data manipulation (querying, adding, updating, and deleting data). Finally, I’ll provide pointers to some popular third party replacements.

One of the big wins for Visual FoxPro’s granddaddy, dBASE II, was that it consisted of both an interactive environment and a programming language. The interactive environment allowed you to directly manipulate data in tables, much like VisiCalc enabled you to work with electronic spreadsheets and WordStar allowed you to do personal word processing. Other socalled ‘database’ tools also allowed you to perform these functions. The difference with dBASE II was that the programming language was really a full-fledged development environment, not just a simple scripting language. You could go way beyond the basic automation of simple tasks and create robust programs with logic trees, database manipulation, and subroutines – once you found your way around the environment. But it was important that you understood how to create and modify tables, navigate your way through those tables, including setting up relations between tables, and make changes to the data in the tables – the timeless add/edit/delete operations. Similarly, you’ll find that you need to know how to do the same types of things with MySQL in an interactive environment before you begin to programmatically work with MySQL databases. There are a number of GUIs that provide interactive access to a MySQL database. The Query Browser is produced by MySQL A.B. themselves, but there are others as well. This chapter will focus on the Query Browser, but at the end, we’ll take a look at several other third party tools as well.

Installation of the MySQL Query Browser Installation varies a bit according to whether you’re running on Windows or Linux.

Windows download Go to http:\\dev.mysql.com, select Downloads, and select the “MySQL Query Browser” link under the “MySQL Tools” heading. You’ll go to a new page that contains all of the Query Browser versions – Windows, Linux, Mac, and so on. Under the “Windows downloads” heading there’s a single link for x86. Clicking on “Pick a mirror” will eventually get you to a link that looks something like this: mysql-query-browser-1.1.17-win.msi

152


(The exact version number may change by the time you read this, of course.)

Linux download Go to http:\\dev.mysql.com, select Downloads, and select the “MySQL Query Browser” link under the “MySQL Tools” heading. You’ll go to a new page that contains all of the Query Browser versions – Windows, Linux, Mac, and so on. Under the “Linux x86 generic RPM (statically linked against glibc 2.2.5) downloads” heading, there are two links, one for nonSUSE RPM installations and the other specifically for SUSE 9.3. Clicking on “Pick a mirror” will eventually get you to a link that looks something like this: Linux (x86, libc6): mysql-query-browser-1.1.18-1.i386.rpm

SUSE LINUX 9.3 (x86): mysql-query-browser-1.1.18-1.suse93.i586.rpm

Again, the exact version number may change by the time you read this.

Windows installation After you download the file to your standard location for such items, right click on the file in Windows Explorer (or your favorite replacement for Explorer), and select “Install.” (If you are running Windows NT or 2000, you may have to install the Windows Installer first, but you probably already have it on your machine.) The MySQL Query Browser Setup Wizard starts, as shown in Figure 1.

Figure 1. The MySQL Query Browser setup starts with the standard Welcome screen.


And proceeds to the License screen, as shown in Figure 2.

Figure 2 . The traditional License screen. Which then is followed by the dialog that allows you to change where the Query Browser will be installed, as shown in Figure 3. I suggest that you put it under the directory where you installed MySQL to begin with.

Figure 3 . Selecting where the Query Browser will be installed. The Wizard then asks you to confirm your choice, as shown in Figure 4.

154


Figure 4 . Confirming your choice of installation location. You can choose either a complete or a custom install, as shown in Figure 5.

Figure 5 . Choosing a complete or custom installation. If you choose Custom, you’ll be prompted to choose which components to install and when, as shown in Figure 6 (somewhat moot with the Query Browser.)


Figure 6 . Selecting the components during a custom install. Once you make all your choices, you’ll be prompted to review them, and then execute the installation, as shown in Figure 7.

Figure 7 . Confirming your choices and then executing the installation.

156


Once complete, you’ll be greeted with the Wizard Completed screen as shown in Figure 8.

Figure 8 . Wizard complete! Once you click Finish, you’re ready to begin using the Query Browser.

Linux installation Installation on Linux varies according to which distribution you’re using. With SuSE Linux, you’ll use YAST to install the RPM that you downloaded earlier in this chapter. With Fedora Core, you have a couple of choices. You can either execute the ‘rpm’ command on the RPM file you downloaded, or use the Add/Remove Software menu option under the Applications button on the task bar. Note that the Linux installation doesn’t use the Windows installer (duh!) and thus you won’t see any of the same dialogs. Rather, it installs invisibly and that’ll be that.

Using the Query Browser In order to use the Query Browser, you have to launch it first. There are several ways.

Starting the Query Browser On Windows, you can start the Query Browser one of several ways: 1. Select the Start | Programs | MySQL | MySQL Query Browser menu item. 2. Select the shortcut that was placed on your desktop. 3. Select “MySQL Query Browser” from the context menu that displays when you rightclick on the MySQL System Tray Monitor. In all cases, you’re running the file


C:\mysql\qb\MySQLQueryBrowser.exe

On Linux, you can start the Query Browser by executing the binary directly via #] /usr/bin/mysql-query-browser

Or you can set up a shortcut to the binary. Right click on the panel, select “Add to Panel,” select “Custom Application Launcher,” and navigate to the binary in /usr/bin. In all cases, you are eventually greeted with the Query Browser login screen, as shown in Figure 9. (From now on, I’ll be showing screen shots in Windows, since that’s likely the OS you’re using, but everything should be essentially the same on both platforms.)

Figure 9 . The login screen for the Query Browser. You need to enter your password, of course. If you know the name of the database you want to initially connect to, you can enter it in the Default Schema textbox, as shown in Figure 10, and that database will automatically be opened in the Query Browser window.

158


Figure 10 . Including a default database when logging in. If you don’t enter a database name, you’ll be scolded with the dialog shown in Figure 11.

Figure 11. If you don’t enter a database name, MySQL will warn you. The options in the dialog are confusing. If you select the OK button, you’re returned to the login dialog in Figure 11. Selecting the Ignore button, however, takes you to the Query Browser, albeit without an open database. (You can choose a database to work with once you’re in the Query Browser.)


Startup Errors This is all fine and good – when everything works as planned. But what happens when something goes wrong? There are two types of errors commonly encountered. The first is if you enter bad credentials in the login dialog. If you enter a bad host, such as the wrong domain name or an incorrect IP address, MySQL warns you with a dialog as shown in Figure 12.

Figure 12 . Error dialog encountered with a bad host. If you click the “Ping” button, the dialog will mutate, as shown in Figure 13, so you can try to determine if the network connection is working.

Figure 13 . Pinging the intended host to check the network connection. In Figure 13, you can see that MySQL isn’t able to reach the inscrutable IP address of 1.2.3.4. Another typical data entry problem is a bad username or password. Again, MySQL warns you with a helpful dialog as shown in Figure 14.

160


Figure 14 . Error encountered with bad credentials. For the example that generated the error shown in Figure 14, I used a bogus username, “herman” on a valid network connection. Clicking the “Ping” button in this case showed that indeed the network connection was good, and thus it must be the credentials supplied that were questionable. See Figure 15.

Figure 15 . Network connection good, credentials bad. Another typical error is entering the wrong schema name. You’re offered a chance to create the schema you entered, as shown in Figure 16.

Figure 16 . Entering a schema name that doesn’t exist will generate an offer to create a new schema with that name.


This can be a handy shortcut to create a new schema, but also allows you a graceful out if you just fat-fingered the database name too quickly. In other words, it’s truly a dialog, in that it gives you options, rather than one of those annoying “OK” monologues.

Parts of the Query Browser The Query Browser has four main parts, as shown in Figure 17.

Figure 17 . The Query Browser has four main components. Unfortunately, they’re not all labeled with the names that the documentation uses, so bear with me while I describe which part of the Query Browser goes with what name.

Query toolbar The top section that stretches the entire width of the form, called the Query toolbar, works similarly to the navigation toolbar in Web browsers, with Back, Next, Refresh, Execute (“go”), and Stop buttons. However, the Query Edit text box is not for Web addresses; rather, it’s where you can enter a SQL command to run against a MySQL database. It can be a pain to spend hours and hours (well, ok, maybe just a minute or two) constructing a long command, and then have to move your hand off the keyboard in order to mouse to the Execute button. Fortunately, there’s a keyboard shortcut for executing a command: Ctrl-Enter. You can put parts of an SQL command on separate lines in the textbox, without resorting to line continuation characters like you do in VFP. For example, the following is perfectly valid:

162


select * from databasename.tablename

Just press Enter where you want the line break to occur – as mentioned, Ctrl-Enter is the keyboard shortcut for execution. You may find the textbox to be too small for some types of queries; the View | Maximize Query Edit (or pressing F11) will create a separate section with an expanded Query Edit text box, as shown in Figure 18.

Figure 18 . The expanded Query Edit text box, displayed via the F11 key. You can resize the height of the Query Edit text box and the Results Area via the sizer bar between the two sections. You might notice that some additional buttons have been added to the right side of the Query toolbar; I’ll discuss those in the section titled “Advanced toolbar” shortly.

Result area Once a SQL command has been executed, the results (or error message) are displayed in the Result area below the left side of the Query toolbar. You’ll notice that the pane has a tabbed identifier (the tab that has a black circle and the word “ResultSet1” in it); this is because you can have multiple result set panes open, and have the result of a SQL command display in one pane or another. This is much like having multiple browse windows open in VFP. The results of the first SQL command you execute is displayed in the “Resultset 1” tab automatically. If you want to create a second tab for your next query, click the folder icon to the left of the “Resultset 1” tab; a new tab labeled “Resultset 2” will appear. You can continue adding tabs as desired. Click on the tab you want results to appear in, and then execute your query in the Query toolbar. Clicking the ‘X’ to the right of the label in a tab closes that tab.


Errors are displayed in a separate message window below the Result area, as shown in Figure 19.

Figure 19 . Errors are displayed in a separate message window under the Results area. In this example, the SQL command is referencing a table in the ‘mysql’ database that doesn’t exist. The Result area also serves as a simplistic editing window on the data in a table. For example, issuing the command select host, user, select_priv, insert_priv, update_priv, delete_priv from mysql.user

displays all user credentials and a few permissions in MySQL’s user table, as shown in Figure 20.

164


Figure 20 . A sample display of results in the Result area. Clicking on the Edit button at the bottom of the Result area enables the fields in the grid to be editable. Double-click on a field and make the desired changes. Once changes have been made, the modified fields will turn color, the Edit button is disabled, and the Apply and Discard Changes buttons (to the right of the Edit button) become enabled.

Sidebar The two panes on the right side of the Query Browser collectively make up the “Sidebar.” The top pane is called the Object Browser; the bottom pane is the Information Browser. It can be handy to temporarily hide the sidebar when you have a result set with a lot of columns and you want to see as many of them as possible. You can display or hide the entire Sidebar via the View | Sidebar menu option. Note that if you have a help topic displayed in the Results area, the Sidebar toggle doesn’t work.

Object browser The Object Browser has three tabs; one for displaying databases, tables, and fields in a treeview arrangement, a second for bookmarking commonly used queries, and a third for scrolling back through previously issued commands, much like you would do in VFP’s command window. The History tab has nodes for various time periods, such as Today, Yesterday, Last Week, and so on, as appropriate for how the Query Browser was used. (In other words, if you didn’t use the Query Browser yesterday, there won’t be a “Yesterday” node.) Figure 21 shows the History tab with a few recent commands.


Figure 21. The history tab of the Object Browser displays recently executed commands. To add a command from the History tab to the Bookmarks tab select the command in the History tab and right-click on the command to display a context menu with a carat (^) as the first menu option. Click on the carat menu option to bring forward the Enter Bookmark Caption dialog, as shown in Figure 22.

Figure 22 . Add your own caption to a bookmark. Enter a caption (such as ‘host’) and click “OK.” The caption is now displayed in the Bookmarks tab, as shown in Figure 23.

166


Figure 23 . Bookmarks are displayed under the folder node in the Object Browser. You can organize your bookmarks by creating your own sets of folders in the Bookmarks tab. Right-click on the Bookmarks node and select “Create bookmark folder.” You can even create multiple levels of folders by selecting a sub-folder before creating a folder. In order to use a command in the Bookmarks or History tabs, select the command and then double-click it to move it into the Query toolbar textbox. From there you can execute the command directly, or edit it first before executing.

Information browser The Information Browser’s purpose is to provide embedded help for building SQL commands in the Query toolbar’s textbox. Clicking on a tab (such as “Syntax” or “Functions”), and then selecting a node displays help topics; selecting a topic will cause that topic’s contents to be displayed in a new tab in the Result area.

Advanced toolbar Doesn’t it annoy you when you go over to someone else’s computer, look at the same program you’re using, and see that their setup looks different. “I don’t have that on mine!” you exclaim. Figure 24 shows the Query Browser with an additional object, the Advanced Toolbar. (So did Figures 18 through 21 and 23.) Experienced users will of course open the View menu, expecting to see an option that says “Advanced Toolbar” simply unchecked, or perhaps a “Toolbar” option that expands into options to display both the Query and Advanced toolbars. But the experienced user would be gravely disappointed.


Figure 24 . The Query Browser with the Advanced toolbar displayed. Instead, in order to display the Advanced Toolbar, you need to select the Tools | Options menu, and check the “Show Advanced Toolbars” checkbox. Now you’re an experienced user. The Advanced Toolbar, despite the trailing ‘s’ in the checkbox caption, is only one object, and consists of three sets of buttons; one set for transactions (Start, Commit, Rollback), one for query management (Explain, Compare), and one for building queries (Select, From, Where, Group, Having, Order, and Set.

Functions of the Query Browser The Query Browser allows you to perform several distinct types of tasks: DDL-type (data definition language) tasks, DML-type (data manipulation language) tasks, and creating scripts. Let’s take a look at each type of task in turn. Note that this is a very brief overview, intended to show you how to use the Query Browser, not to give you an in-depth lesson on SQL. There are several excellent books that cover this topic well, including Joe Celko’s “SQL for Smarties” and our own “Taming Visual FoxPro’s SQL” by Tamar Granor.

Data Definition tasks Data definition tasks include creating and deleting databases as well as creating, modifying, and deleting tables and indexes. In order to create a database named ‘mystars’, execute the command create database mystars

168


You’ll see a new ‘mystars’ folder in the Database Root folder (for example, C:/Program Files/MySQL/MySQL Server 5.0/data). Under the ‘mystars’ folder, you’ll find a new file, db.opt – regardless of whether you’re using MyISAM or InnoDB. If ‘default storage’ is set to ‘innodb’, you’ll also see that the ibdata1 file in the InnoDB storage location has been modified. (You might have changed it during installation, or during configuration – see the InnoDB Parameters tab in the Startup Variables node of the MySQL Administrator.) You’ll also see a confirmation message displayed under the Result area that says something like “1 row affected by the last command, no resultset returned.” This confirms that the command was successful, but that you weren’t doing a SELECT that would produce results. Once a database is created, you can see it in the Object Browser pane in the Query Browser. If it doesn’t show up immediately, right-click in the pane and select the “Refresh” menu option. (The green Refresh button in the toolbar is used for re-executing queries; it doesn’t apply to the Object Browser pane.) You can also issue the use mystars

command. A database by itself isn’t very interesting – typically you’d want tables too. Add a table to the database with the following command: create table mystars.rockstars (iidrockstars int, cnaf char(20), cnal char(20))

You can avoid having to include the ‘mystars.’ database identifier if you select the database first, say, by clicking on its node in the Object Browser. You can now drill down into the database and table structure in the Object Browser, as shown in Figure 25.


Figure 25 . You can drill down to the field level of a table in the Object Browser. In order to get rid of a table in database, issue the following command: drop table mystars.rockstars

and to get rid of an entire database, issue the drop database mystars

command. The ‘alter table’ command, with all its multitude of op tions, allows you to modify existing tables to your heart’s content.

Data Manipulation tasks Data manipulation tasks include adding, editing, and deleting records and the contents of fields as well as selecting records from one or more tables in a database. Create the “mystars” database and the “rockstars” table again with: create database mystars create table mystars.rockstars (iidrockstars int, cnaf char(20), cnal char(20))

Add a record to a table like so: insert into mystars.rockstars

170


(iidrockstars, cnaf, cnal) values (1, "Eddie", "Van Halen") insert into mystars.rockstars (iidrockstars, cnaf, cnal) values (2, "Alex", "Van Halen") insert into mystars.rockstars (iidrockstars, cnaf, cnal) values (3, "Michael", "Anthony") insert into mystars.rockstars (iidrockstars, cnaf, cnal) values (4, "Sammy", "Hagar")

Edit the record like so: update mystars.rockstars set cnal = ‘Bertinelli’ where iidrockstars = 1

Delete a record from a table like so: delete from mystars.rockstars where iidrockstars = 3

Select just the Van Halens from the table like so: select * from mystars.rockstars where cnal = "Van Halen"

(Remember that we just changed Eddie’s name a few commands ago!)

Stored Procedure (script) tasks What the Query Browser refers to as ‘scripts’ are also known as stored procedures. MySQL 5.0 and higher support stored procs, and the Query Browser is an easy way to work with them. Stored procedures are covered in Chapter 21, Getting Started with Stored Procedures.

Other Tools There are a number of third party tools that provide similar functionality as the native Query Browser, some free, and some commercial. If you’re interested in trying out others, here are several of the most popular alternatives. Navicat www. navicat.com Windows, Linux, OSX 30 day free trial


MySQL Maestro http://www.sqlmaestro.com/index.php Windows only 30 day free trial

SQLyog http://www.webyog.com/ Windows only Free version MySQL Front http://www.mysqlfront.de/ Windows only Free for 30 days, then aprox $35

A couple of other general tools you can use to work with MySQL data include phpMyAdmin (http://www.phpmyadmin.net) and xCase (http://www.xcase.com/ ).

Conclusion/Summary The Query Browser allows you to work interactively with MySQL data in a GUI environment much like you’re used to doing with native Visual FoxPro tables. In fact, it’s even easier than working with MySQL data interactively inside VFP because the Query Browser handles the connections for you, while in VFP, you need to set up and manage the connection handle yourself. Updates and corrections to this chapter can be found on Hentzenwerke’s Web site, www.hentzenwerke.com. Click “Catalog” and navigate to the page for this book.

172


Chapter 9: Under the Hood: Where MySQL Keeps Its Data 173

Chapter 9 Under the Hood: Where MySQL Keeps Its Data When MySQL starts up, what happens under the hood? How does it know where the databases are? How does it know who can connect to the system, and, once connected, which databases they can access, and what they can do to a database and the data inside? While this topic has been broached in Chapter 7, “Configuration,” there’s plenty more still to come regarding how MySQL finds its data and how access to that data (privileges) is stored and structured. This chapter will finish up what we started in Chapter 7 so you know how MySQL drives, and is driven, by its privilege data stores.

Some folks are comfortable knowing their data is being handled in a big black box ‘somewhere’ and assume that it’ll always be there for them. Other folks want to get under the hood, at the very minimum to have an understanding of what’s going on, even if they’re never really going to mess with it themselves. Fox developers generally fall into the latter camp.

Types of MySQL data There are generally three types of SQL database data. The first is the data itself – the databases, indexes, and so on. These are the file(s) that contain your customer records, lists of pets or housing records, parts inventory, or whatever your database system tracks. The second is the description of your database – what is the name of the database, the names of the tables, the fields in each table, the indexes, and the expressions those indexes are based on, and so on. This is referred to informally as ‘the data about the data’, and more formally as ‘metadata’. Some SQL databases keep all of this information in a central location; others keep it with the actual data. The third type of data has to do with users and rights, or, as MySQL refers to them, privileges. Who is allowed into the system? Once in, is that user allowed to access a specific database? And if so, what functions are they allowed to perform during that access? Let’s take a look at where and how MySQL addresses each of these.

Where is your data? Suppose you have a software application that tracks telephone billing records, the “Telephone Billing System.” You have tables for customers, telephone calls, invoices, calling plans, payments, and so on. And you’re using MySQL to store all of this data. Where are the tables that store your customer, telephone call, invoice, and other data? The answer is... it depends. Specifically, it depends on what type of database storage engine you use; MyISAM or InnoDB. However, MySQL finds out the location of your data the same way regardless of the type of engine – via the MySQL configuration file (my.ini in Windows and my.cnf in Linux.)

174


Windows On Windows, when the MySQL service starts up, the executable looks for the configuration file, my.ini, in the location specified during startup. For example, in Windows, the configuration file is identified in the “Path to executable” read-only textbox in the General tab of the MySQL Properties dialog (opened via the Services applet), as shown in Figure 1.

Figure 1. The location of my.ini is part of the “Path to executable” read-only text box. Since the entire string won’t display comfortably in the text box, here it is (wrapping onto two lines): "C:\Program Files\MySQL\MySQL Server 5.0\bin\mysqld-nt" --defaultsfile="C:\Program Files\MySQL\MySQL Server 5.0\my.ini" MySQL

The my.ini contains the following parameters that are of interest to us in our quest to find our data. •

default-storage-engine: This parm defines what the default storage engine is, which will be useful shortly. The two choices we’re concerned with look like this:


default-storage-engine=myisam

or default-storage-engine=innodb •

basedir: Where MySQL is installed – the top level directory for MySQL. This parm is held in memory and paths to other programs and locations are resolved relative to this directory. Here’s an example:

basedir="C:/Program Files/MySQL/MySQL Server 5.0/" •

datadir: The path to the database root directory. We’ll use this for two different purposes. First is that it’s the location of the MySQL privilege data (covered shortly). Second is that it’s the location of MyISAM databases. Here’s an example:

datadir="C:/Program Files/MySQL/MySQL Server 5.0/Data/"

So, in our example of the Telephone Billing System application, you would have a directory with the name of the application’s database, say, ‘telebill’. Underneath that directory, you’d have all of the tables and related files in the telebill database, like so: "C:/Program "C:/Program "C:/Program "C:/Program "C:/Program •

Files/MySQL/MySQL Files/MySQL/MySQL Files/MySQL/MySQL Files/MySQL/MySQL Files/MySQL/MySQL

Server Server Server Server Server

5.0/Data/telebill" 5.0/Data/telebill/customer" 5.0/Data/telebill/plan" 5.0/Data/telebill/invoice" 5.0/Data/telebill/payment"

innodb_data_home_dir: If you’re using InnoDB storage, this is the path to the InnoDB database. Here’s an example.

innodb_data_home_dir="E:\wsdb\mysqldata\"

This value is set up during the Windows installation (see Figures 24 and 26 in Chapter 3). Where are the tables? By default, InnoDB databases are contained in one big ol’ file, like so: "E:\wsdb\mysqldata\ibdata1"

but if you split the database into multiple tables, the tables would all be under the \wsdb\mysqldata directory.

Linux The MySQL Instance Manager is started with /etc/init.d/mysql script on Linux. As of MySQL 5.0.10, the MySQL Instance Manager reads and manages the /etc/my.cnf file on Unix. The default option file location can be changed with the --defaults-file=file_name option in the script. What this means is the /etc/my.cnf file is again where we’re going to find MySQL is expecting our data.

176


•

datadir: The path to the database root directory. Again, this path will be used for two different purposes. First is it’s the location of MyISAM databases. Here’s an example:

datadir=/var/lib/mysql

Interestingly enough, while the Windows installation gives you the option to put your InnoDB databases somewhere else (maybe realizing that burying your database files in the bowels of “Program Files” isn’t a very good idea?), the Linux installation just goes ahead and puts both your MyISAM and InnoDB files in the same place – /var/lib/mysql, which is where they should be. (Microsoft has traditionally given short shrift to our data, and I can see why the MySQL designers didn’t think that “My Documents” was an appropriate place for databases.) Thus, again using the Telephone Billing System example mentioned earlier, you would have a directory named the same as the application’s database, say, ‘telebill’. Underneath that directory, you’d have all of the tables and related files in the telebill database, like so: /var/lib/mysql/telebill /var/lib/mysql/telebill/customer /var/lib/mysql/telebill/plan /var/lib/mysql/telebill/invoice /var/lib/mysql/telebill/payment

Or for InnoDB databases, you’d have /var/lib/mysql/telebill" /var/lib/mysql/telebill/ibdata1

just like with Windows.

Where is the MySQL metadata? MySQL does not store its metadata – the descriptions about which databases belong to the server, what tables are in each database, and what’s in each table – in a separate set of tables. Rather, the metadata is stored with the data itself.

MyISAM Each table in each database consists of three files, each with the same filename but different extensions: .frm: The table format file, which contains information about the table structure .myd: The table data .myi: The table indexes Thus, MySQL simply looks in the %datadir directory for a list of databases (as identified by the directory names), determines the tables in the database by looking for all files with .frm extensions, and then, finally, parses the .frm files for the contents of each table.

InnoDB The InnoDB data structure is all self-contained inside the ibdata1 file.


Sometimes the answer just isn’t very complicated.

Where is the MySQL privilege data? I’ve alluded to ‘files that describe who is allowed to do what’. More specifically, when you connect to MySQL, the server determines who you are, both by your user login and the host from which you are connecting. This is because MySQL servers can be accessed throughout the Internet – and different hosts could potentially have the same user login, and the same login might have different privileges from different hosts. Furthermore, once you do connect, MySQL grants you the ability to perform tasks based on who you are and what you want to do. For example, there are likely many users named ‘bob’ out there in Internet-land. You wouldn’t want any old ‘bob’ to get into your database, so you need to restrict just ‘bob’ from ‘example.com’ as well as allowing ‘bob’ from ‘localhost’ to connect to the server. Additionally, ‘bob’ from ‘localhost’ may well have a wide range of capabilities, including creating and dropping databases, modifying database structures, and so on. However, ‘bob’ from ‘example.com’ may have a much more limited range of allowable tasks, such as select, insert, and update. You can set up MySQL to have multiple accounts for bob, one for the ‘localhost’ host, and a second for the ‘example.com’ host. The MySQL privilege files control all these capabilities. Let’s take a look at them in more detail. There are over 50 files in the mysql directory, 54 to be exact. They are: columns_priv.frm columns_priv.MYD columns_priv.MYI db.frm db.MYD db.MYI func.frm func.MYD func.MYI help_category.frm help_category.MYD help_category.MYI help_keyword.frm help_keyword.MYD help_keyword.MYI help_relation.frm help_relation.MYD help_relation.MYI help_topic.frm help_topic.MYD help_topic.MYI host.frm host.MYD host.MYI tables_priv.frm tables_priv.MYD tables_priv.MYI time_zone.frm time_zone.MYD time_zone.MYI time_zone_leap_second.frm

178


time_zone_leap_second.MYD time_zone_leap_second.MYI time_zone_name.frm time_zone_name.MYD time_zone_name.MYI time_zone_transition.frm time_zone_transition.MYD time_zone_transition.MYI time_zone_transition_type.frm time_zone_transition_type.MYD time_zone_transition_type.MYI user.frm user.MYD user.MYI proc.frm proc.MYD proc.MYI procs_priv.frm procs_priv.MYD procs_priv.MYI user_info.frm user_info.MYI user_info.MYD

The privilege files are made up of 18 tables; each table has three files associated with it. The FRM file is the table definition file, the MYD file holds the actual data, and the MYI file is for indexes. Some are more interesting than others. We’ll ignore the ‘help_*’ and ‘time_zone*’ files; the contents of the former are obvious, the latter have to do with how the MySQL server maintains time zone settings. (See Section 5.11.8, “MySQL Server Time Zone Support” in the online help for details.) So that leaves 9. •

The user table contains the user accounts inside MySQL. There are columns that define the account – user, password, and host – and columns that define the privileges for the account – select_priv, insert_priv, delete_priv, and about twenty more. This table provides the initial ‘line of defense’ to MySQL. Connections are allowed or rejected based on matches of credentials presented (username, password, host) against the same columns in the user table (user, password, host). Once access has been granted, the values in the rest of the columns in the table (such as select_priv) define global privileges. The default value for all privileges is ‘N’, so a new account added to the system doesn’t automatically get to do anything until the person creating the account explicitly decides to allow it to. This table is where accounts for ‘[email protected]’ and ‘bob@localhost’ are stored. As you can see, the other columns in this table grant access to specific functions for those two accounts.

•

The user_info table contains additional fields for a user record. Fields include full name, email, and contact information. Fun, but not directly related to privileges.


•

•

•

•

The db table provides more granular permissions, defining which users – from which hosts – can access which databases. As with the user table, additional columns define tasks that can be performed, such as select_priv, insert_priv, and so on. The host table is similar to the user table, in that it provides the ability to restrict access to the MySQL environment, but instead of on a host/user basis, restrictions can be set up on a host/database basis. This table is used in conjunction with the db table when you want to have multiple hosts to work with a specific row in the db table. There are columns for both host and database (as just mentioned) as well as enumerated columns for select, insert, update, delete, create, and another dozen or so types of privileges. The server can also be set up to provide privilege control at the table and column levels. The tables_priv and columns_priv tables are used for this purpose. The func and proc tables provide the list of stored procedures. Access to these routines is granted by entries in the procs_priv table.

My purpose here was to give you an overview of how MySQL privileges work. For more details, see 5.8.2. “How the Privilege System Works” in the online help.

Moving your Windows data out of Program Files The MyISAM examples in this chapter show MyISAM databases located deep in the bowels of the Program Files directory on C while the InnoDB databases are located on a separate drive. If you want to move your MyISAM databases to a separate drive, you need to take several steps. First, create a separate data directory, say, e:/mysqldata. Second, shut down the server so you get access to your data files. By the way, if you’re thinking of making changes to your my.ini file manually (not through the MySQL Administrator), keep a backup of the last good my.ini. Third, move ALL files under MySQL Server 5.0/Data to this new directory, including the mysql directory. Fourth, change the database root in my.ini to read: datadir="e:/mysqldata/"

And finally, start your server again. If you try to create an e:/mysqldata dir, and then just move the telebill (or customers and pets) directory (but not the mysql directory), and point the database root to e:/mysqldata, bad things will happen. Specifically, MySQL will look in e:/mysqldata for the metadata files (those that are still in the MySQL Server 5.0/Data/mysql directory, not find them, and terminate with an unfriendly error. Words of experience here! If you wanted to have data scattered in many different locations, you would have to start different instances of MySQL, with each instance pointing to its own configuration file. If you make a mistake (such as specifying the wrong option or just making a typo), MySQL won’t start up again. If you use the MySQL Administrator tool, backup copies are automatically made for you. (See Chapter 7 for details.)

180


Conclusion/Summary Initially it can be a little confusing to determine where MySQL’s various data files are and how they interact. When the answer isn’t obvious to you after a 30 second inspection, it’s tempting to assume all sorts of complexity. Sometimes, though, a cigar is just a cigar. You just have to look at the configuration file first. Now let’s move on to creating data sets from scratch. Updates and corrections to this chapter can be found on Hentzenwerke’s Web site, www.hentzenwerke.com. Click “Catalog” and navigate to the page for this book.

Chapter 10: Creating Data Sets from Scratch 181

Chapter 10 Creating Data Sets from Scratch This chapter covers two separate topics. First, I discuss the differences between Visual FoxPro data types and MySQL data types, and how to choose which type you should use when creating a brand new table. Second, I cover how to create an empty database and how to set all of the features of the table and field editors. For those of you who are worried about that big chunk of data sitting in an existing database, do not fret! I’ll cover conversion from text files to MySQL in the next chapter, and from DBFs to MySQL in the chapter after that.

While existing applications already have a database filled with data, at some point in time, you’ll probably need to create an empty database and put some tables in it. Here’s how to do it the right way, from scratch. Before I get into the nitty-gritty of discussing data types and creating structures, I’m going to provide a high level review of database design, so we’re all starting at the same place. I don’t want to make assumptions about knowledge that you might be missing. Note that while database design is very much a science, there are many places where the designer has a great deal of latitude, and other places where there is more than one equally valid choice. Since this is my book, I’ll discuss what I think are, in my opinion, the best choices, and the reasoning behind those choices. You are, of course, free to agree or go your own way.

Best practices in database design A “best practice” is a generally accepted technique for performing an operation. The technique might be a best practice because it’s technically superior (faster, more reliable, less prone to error) than alternatives. A technique might also be a best practice because the community has chosen this one, in lieu of other choices, simply so that everyone can standardize on the same way to do that operation.

Relational database theory While I’m sure that you have a basic grounding in relational database theory as it relates to normalizing data, I’m going to reiterate a few key points that I’ll rely on as we design our tables. If you find a lot of this is new to you, you may want to do some additional reading on the subject. Here are a few good resources: • • • • •

•

The Practical SQL Handbook , Bowman Handbook of Relational Database Design, Fleming, vonHalle http://www.tedroche.com/Present/2004/RocheDataDesign.pdf Database Design For Mere Mortals, Mike Hernandez (ISBN: 0201694719) Designing Effective Database Systems, Rebecca M. Riordan (ISBN: 0321290933)

http://en.wikipedia.org/wiki/Database_normalization

182


Naming conventions There are two choices as far as naming conventions go – one camp likes to use naming conventions for databases, tables, fields, and indexes. The other doesn’t. Here are some pros and cons. After you read through this, you may change your mind and move to the other camp – that’s OK, regardless of which way you go. The important thing is to be consistent! If you’re going to use a naming convention, use it all the time, and apply it consistently. If you’re not, then don’t use it just part of the time – you’ll drive the folks mad who follow in your footsteps, and unless that’s your intent, well, you know.

Database I prefer to use short words for databases because I’m going to be using that string of characters a lot. I’d rather type “iasds” than “internationalautomotiveselectiondatabasesystem.”

Tables I use a word that describes what is represented by the contents of a single row in the table. For example, my tables are named “customer,” “pet,” and “car,” not “customers,” “pets,” and “cars.” Other folks prefer to go the opposite way – “customers,” “pets,” and “cars.” Please don’t mix and match like this, though: “customers,” “pets,” “car,” and “invoice.”

Fields Many folks in the VFP camp are accustomed to using Hungarian notation not only for naming variables in their code, but also their field names in tables. That’s not as commonly followed in other communities of developers, such as MySQL. There are several arguments for not using Hungarian. The first is that simply using a prefix for identifying a data type doesn’t force that column to contain that data type. You could have a field named “cInvoiceNo” but its contents might be all integers, or a field named “nInvoiceNum” might contain character data. (And it’s not always the developer’s fault. They might have initially been told invoice numbers are all numeric, thus prompting a numeric field type – and six months later, after boxes of code had been written, all referencing the ‘nInvoiceNum’ field name, someone decides the field needs to be able to handle alphabetic characters as well. So the field type is changed, but not the name. Ugh.) Another argument insists that a field’s data type should be obvious by its name, and should not be dependent on a randomly chosen prefix. “Birthdate” is obviously a date, “First_name” is obviously a character field, and so on. “If you do math with it, it’s a number, otherwise, it’s a string” goes the explanation. A third argument is that there are sufficiently more data types for MySQL (or other backend databases) that it becomes extremely difficult to come up with consistent and logical prefixes for each data type. For example, in Visual FoxPro, there are just a few types of numeric-ish data types: numeric, integer, double, float, and currency. In MySQL, however, there are a lot: bit, tinyint, smallint, mediumint, int, float, double, and decimal. There are also duplicate names for several of these data types: dec = decimal, int = bigint, and so on. Obviously, I don’t agree with the above arguments against using Hungarian. While it’s true that the name doesn’t force a field to contain that type of data, that’s true also for “birthdate” and “first_name.” Furthermore, there are a number of common situations where the field name doesn’t clearly identify the data type; “id_number,” “code,” “invoice number” – these all might be numeric or integer values in some systems.


Even more so, using a nebulous yardstick (‘if you do math with it’) to determine type requires the developer to intimately know the data. If you saw a field in an application that was labeled ‘counter’, what data type would you figure it was? Obviously an integer. What if the application was a furniture store management application? Hmmmm, maybe not so obvious after all. Maybe the field holds a code for the type of counter. Or maybe it was a logical field that reflected whether or not a counter was included in a particular assembly. Yes, in a perfect world, the data type would be obvious from the context in which the ‘counter’ field lived. Let me know when you come across one of those perfect worlds.

Do you carry your VFP habits over to the MySQL world? Ultimately, this is the big question. I do, for a couple of reasons. First, many of my applications go from VFP tables to MySQL, and by keeping the table and field names the same, there’s less work involved than if I was to completely rename everything. Second, having fields named with Hungarian notation always made sense to me. A field named “code” might be numeric or character; a field named “ccounter” clearly contains a text string. (If a text field named “ccounter” was used to increment a counter, then there’s a far larger problem to deal with.) But that’s just me. As I said, regardless of which way you choose, be consistent about it.

Keys There are essentially two types of keys: primary and foreign. Strictly speaking, there are others, such as candidate and compound keys, but we’re just concerned with primary and foreign for the time-being.

Primary keys A primary key serves as the unique identifier for a row in a table. It ensures that each row in a table can be selected without ambiguity. Technically, primary keys can be made up of one or more columns, such as last name + address + phone number. However, best practice states that the primary key’s sole purpose is to identify the record in a table. It should not have any meaningful information to users (like Social Security Numbers or ZIP codes). It should be an integer and only span one column. This type of primary key is called a surrogate key. A surrogate key filled with integers will provide best performance for retrieval and joins between tables. As you can imagine, it’s less work for the computer to deal with the integral value “410348” than, say, a 16 byte GUID (67C4D234-5AAD-402C-82A1-D512FF1D2406) that’s stored as a character string. Many database developers are tempted to use one or more fields that contain live data to make up their primary key, such as invoice_number or zip_code. “I don’t want to waste the storage space used by a separate meaningless primary key field,” they’ll say, or “This data will never change!” The disk space saved is trivial, especially these days, and even for very narrow tables with just a couple of fields, the storage requirements just aren’t significant. More importantly, though, as soon as your primary key uses data that has meaning, you can bet that a new requirement for the application is going to make your life miserable. The here-to-fore unique invoice number gets munged when the company buys another company, and their invoices have to be merged with yours – and their invoice numbers either aren’t compatible, or worse, are compatible – down to having used the same values! Suddenly the unique invoice number “1057002” has a duplicate in the table, from the other company.

184


Even data that is very clearly going to be unique forever – such as a zip code – can be changed. How many systems that relied on a unique five digits zip code had to be changed when the post office introduced zip plus four?

Foreign keys A foreign key is a field in a table that points to a primary key in another table. It is used to relate the two tables. For example, suppose you have a customer table with a primary key. You can use the customer table’s primary key value as a pointer in an invoice table so the invoice table can point to which customer the invoice belongs to. (Most of you probably think that this is pretty basic information, but I have run into folks who (1) thought they invented the concept themselves, or (2) worse, weren’t aware of the concept at all, and duplicated data throughout their database. You just never know...) Since a foreign key in a table points to a primary key in another table, the foreign keys should also be the same data type as primary keys: single columns filled with meaningless integers.

Time stamps A time stamp is a datetime-type field that identifies when a row in a table was last touched. I put four time stamp-related fields in every table I create – a field for the user ID of who created the record, a second field for the time stamp that records the creation of the record, and two more that identify the who and when of the last update to the record. (If I needed to know more than just the creation and last edit of a record, it’s time to go for a full audit trail.) The first two only get touched once – they get stuffed into the table when the record is created. The second pair is updated whenever the record is modified. In the event that the application is set up to flag a record as ‘deleted’ (similar to Visual FoxPro) instead of absolutely removing the record, the second pair is updated to reflect the setting of the deletion flag. (Unlike VFP, when you issue a DELETE command on a MySQL table, the records in question are gone forever. Thus, if you want the ability to recall a deleted record like you can do in VFP, you need to include a field to act as the deleted() flag in your MySQL table, and then build application logic around it. You may want to use two fields – one to indicate when the record was deleted, and a second to record who deleted the record.)

Overloading tables The term “overloading” means using a table for more than one type of data. The classic example is using a lookup table to hold a variety of values that are related to foreign keys. For example, instead of having one table to hold country abbreviations: AL CA DE MT NF SE SK TN US

Albania Canada Germany Malta Norfolk Islands Sweden Slovakia Tunisia United States

and a second table for state abbreviations:


AL CA DE MT TN

Alabama California Delaware Montana Tennessee

and a third for province abbreviations: AB BC NF SK

Alberta British Columbia Newfoundland Saskatchewan

you use a single table that combines all of the lookup tables. This table would have three data fields, for the type of entity (Province, State, or Country), the abbreviation, and the full name (and, of course, the primary key and the audit trail columns, like added and updated flags). It would look like this: Prov State Country Prov State Country State Country Country State Prov Country Country Prov Country State Country Country

AB AL AL BC CA CA DE DE MT MT NF NF SE SK SK TN TN US

Alberta Alabama Albania British Columbia California Canada Delaware Germany Malta Montana Newfoundland Norfolk Islands Sweden Saskatchewan Slovakia Tennessee Tunisia United States

The advantage is that you can consolidate what may end up being dozens (or even hundreds) of little tables, some with only a few records, into one table. Not only is your ERD (entity relationship diagram) a lot simpler, you only need to write one interface for maintenance of all of the data, instead of writing from scratch or cloning an existing form for every individual lookup table.

Choosing data types MySQL has a large number of data types, some of which are SQL standard and others that are an extension to the standard. In some cases, the data type for a field will be obvious, while in others there will be multiple options that may appear to be equally valid. The importance of choosing the perfect data type versus one that’s ‘good enough’ for each field depends on your situation. The difference between choosing the optimal type and a suboptimal type won’t matter much in a 5,000 row table, but will be extremely important in a 50,000,000 row table.

186


Available data types in MySQL There are three general groups of data types in MySQL: • • •

Numeric Date/Time String

Here’s a quick overview. There are a lot of specific details; check out section 11, “Data Types,” in the on-line help for nuances.

Numeric types Numeric types consist of the following standard SQL data types: •

•

•

INTEGER (also known as INT): 4 bytes, values ranging from -2,147,483,648 to 2,147,483,647 (signed) or 0 to 4,294,967,295 (unsigned) SMALLINT: 2 bytes, values ranging from -32,768 to 32,767 (signed) or 0 to 65,535 (unsigned) DECIMAL (also known as DEC): Similar to VFP. Example for the field named ‘amount’:

amount DECIMAL (5,2)

can store from -999.99 to 999.99 •

•

NUMERIC: Same as DECIMAL in MySQL FLOAT: 4 bytes, holds approximate numeric values to 23 places. Example for the field named ‘amount’:

amount FLOAT(7,4)

results in 999.9999 •

•

REAL: Same as DOUBLE unless REAL_AS_FLOAT is SQL specified DOUBLE PRECISION: 8 byte, holds approximate numeric values to 54 places

MySQL also supports additional data types: BIT: Used to store bit-field values (e.g. 10001000) – you can store the equivalent of eight logical fields in one BIT field, using the XOR function to test the value of a specific place •


•

•

•

TINYINT: 1 byte, ranging from -128 to 127 (signed) or 0 to 255 (unsigned) MEDIUMINT: 3 bytes, values ranging from -8388608 to 8388607 (signed) or 0 to 16777215 (unsigned) BIGINT: 8 bytes, values ranging from -9,223,372,036,854,775,808 to 9,223,372,036,854,775,807 (signed) or 0 to 18,446,744,073,709,551,615 (unsigned)

Using the “UNSIGNED” keyword as part of a data type definition shifts the range from a negative to positive span (such as -128 to 127 for TINYINT) to an all positive range (0 to 255 for the aforementioned TINYINT data type.) If you attempt to insert a negative value into a field that is defined as “unsigned,” you’ll get an ‘out of range’ error.

Date/Time types There are five data types for representing time-related data. These fields each have a range of legal values (MySQL engine will reject illegal values, and what constitutes ‘illegal’ varies from one version of MySQL to the next) as well as a specific “zero” value for use when you try to specify an illegal value. •

DATETIME: Used for storing a date plus time of day (in 24 hour format) in a single field. Examples:

1980-1-1 21:30:15 1999-12-01 00:00:00 0000-00-00 00:00:00

Use quotes to delimit a datetime value: insert into rockstars (cnaf, cnal, dob) values ("Paul", "McCartney", "1942-6-21 16:10:58")

If you stuff a date but not a time in a datetime field, the time is set to 00:00:00. Remember to convert VFP times with AM/PM to 24 hour format with SET HOURS TO 24. •

DATE: Used for storing just a date. Examples:

1980-1-1 0000-00-00 •

TIMESTAMP:

188


The same as a datetime, except it can be automatically updated upon the addition or modification of a record. Note that you have to provide specific code to do so; simply identifying a field as a timestamp doesn’t mean it’s updated automatically. See the section on “Working with time stamps” later in this chapter for details. Examples: 1980-1-1 21:30:15 0000-00-00 00:00:00 •

TIME: Data type that just stores time, in 24 hour format. Examples:

6:10:00 (early in the morning) 18:31:52 (dinnertime news just finished) 00:00:00 •

YEAR: Data type specifically for holding a year. Range is 1901 to 2155. Examples:

1944 2141

Can specify as number or string. insert into rockstars (yyeartour, yyearfirstalbum) values ("1961", 1962)

You can also use ‘0000’ as an empty value. A number of additional notes apply to these data types: A variety of punctuation symbols can be used. For example, 1980/1/1 and 2001-12-31 1610-04 (10 minutes past 4 pm on 12/31 of 2001) are both legitimate. Using a string that has no delimiters is also valid, as long as the date the string evaluates to is valid. Thus, 19951231 (last day of 1995) is OK but 19950231 (February 31 st) is not. The DATE, DATETIME, and TIMESTAMP fields can take a two digit year instead of a four digit year. However, as you can imagine, there are some constraints and assumptions made. First, two year dates between 70 and 99 are converted to four year dates between1970 and 1999 while two year dates between 00 and 69 are converted to four year dates between 2000 and 2069. Dates must be given in YY-MM-DD format. The American convention of MM-DD-YY is not valid. Note that if you’re using a pre-5.0 version of MySQL, the rules for dates and times are different. For example, the engine is not as strict when it comes to validation.

String types There are 8 string types, most of which will be familiar to VFP developers. CHAR, VARCHAR: •


These data types are both declared with the maximum number of characters to be stored in the field. Char can be 0 to 255. If data less than the declared length is stored, the data is padded to the right. Varchar fields are variable length. 0 to 65K. Varchar data is stored without padding spaces, albeit with a couple of extra bytes that describes the record length. If the data for a field is fairly similar in length, or the data size is small (say, 10-25 characters), use a char field to improve processing speed; if the size of the contents of a field varies greatly, use a varchar field to save space. Examples: postal CHAR(10) city VARCHAR(45) •

•

•

•

•

BINARY, VARBINARY: These two are just like Char and Varchar, except the data stored isn’t simply characters, it’s any type of binary character. For instance, you couldn’t store a line feed in the middle of a char field, but you could in a binary field. BLOB: A blob, an abbreviation for Binary Large OBject, has four types: tinyblob, blob, mediumblob, and longblob. They differ in terms of the amount of data the field can hold. Blobs have no character set, and thus can hold any type of data. TEXT A Text field, similar to a VFP memo field, also has four types that correspond to the four blob types: tinytext, text, mediumtext, and longtext. Text fields differ from blobs in that they have a character set. ENUM: The enum data type is used for fields where the domain (the set of allowable values) is relatively small and can be chosen from a known set. For example, an “invoice_status” field that only has values of “PAID,” “OPEN,” and “PENDING” would be a good candidate for an enum data type. Enums are string data. An enum can have a maximum of 64K elements. SET: The Set data type is similar to the Enum in that it provides a pre-defined list of elements, but the way they’re stored and handled is different. Furthermore, a Set field can have at most 64 elements.

You may be wondering what the difference between varchar and text data types are, or, more specifically, why you’d use one versus the other. Text (and BLOB) columns cannot have default values and you must specify an index prefix length for text columns (just like you can’t index on the entire contents of a VFP memo field.)

Comparing MySQL data types to VFP’s choices VFP 9 has 18 different data types. Most of them translate directly to MySQL types. A couple have more than one mapping to MySQL, but a couple don’t have direct replacements. Here’s how to map VFP to MySQL data types.

190


Numeric Currency maps to Decimal Double maps to Double Float maps to Float Integer maps to Integer (4 bytes) Numeric fields in VFP can range from +/-999,999,999,9 E+19, and so map to decimal or double, depending on what was stored in the VFP numeric field. Note that the VFP numeric type includes the decimal point in the definition while the MySQL numeric type does not! (For example, VFP N(5,2) stores 99.99 while MySQL N(5,2) stores 999.99.) The VFP Integer auto-increment data type maps to a MySQL Integer data type, and the MySQL table will have to be defined with this field as the primary key. (This is covered later in the section, “Creating Primary Keys.”) • • • • •

Date/time • •

Date maps to Date Datetime maps to Datetime

String • • • • • • • • •

•

Character maps to Char Character (binary) maps to Binary Memos with a max of 65K characters map to Varchar, otherwise, to Text Memos (binary) with a max of 65K characters map to Varbinary, otherwise to Blob Varbinary maps to Binary or Varbinary Varchar maps to Char or Varchar Varchar (binary) maps to Binary or Varbinary Blob maps to blob General maps to blob but may not transfer properly, as VFP general fields are supposed to be used to reference an OLE object, which may or may not be applicable, depending on which operating system(s) your application will be running on. (OLE only works with Windows.) Logical fields become tinyint(1). The MySQL Connector considers Tinyint(1) as a replacement for a logical type, so you will see ‘true’ or ‘false’ as values of a TINYINT(1) instead of the à priori numeric values.

Making the decision It’s not always clear which type to use. For example, choosing a memo field’s replacement type in MySQL – if you used a memo field to hold an address that could contain multiple lines (but would never be more than 500 characters, say), then a MySQL varchar(500) field would be fine. If, however, you’re storing reams and reams of text in that memo field, then a varchar() field might not be appropriate. You have to know your data.

Creating your database from scratch You can create a database, the tables in it, and the keys and indexes for each table, either through a GUI tool like the Query Browser, or via SQL commands. In the following sections I


describe how to use the Query Browser first, and then list the SQL commands that perform the same operation.

Creating a database You can use the Query Browser to create a new database. Right click in the Schemata tab and select the “Create New Schema” menu option from the context menu as shown in Figure 1.

Figure 1. Creating a new database in the Query Browser. You’re prompted for the name of the database, as shown in Figure 2.

Figure 2 . Naming your database. Once you click the “OK” button, your new database shows up in the Schemata tab. (If it doesn’t, right-click in the pane and select the “Refresh” menu option.) Behind the scenes, a new directory with the name of your database is created in the “database root” location specified in the configuration file, and a file named “db.opt” is placed in that directory. These two things happen regardless of the type of default storage engine specified in the configuration file. As mentioned in Chapter 9, the existence of this directory is how MySQL knows about the database at all.

192


You can also create a database via SQL commands. Enter the following command (either in the Query Browser or through the MySQL monitor.) create database superheroes

and the same two things happen behind the scenes. You can also delete a database via the right-click menu in the Schemata tab or by issuing the command drop database superheroes

Note that you can delete a database that has tables (and data!) in it, which makes this command rather dangerous.

Creating tables A database without tables isn’t very interesting. Downright dull, some would say. Much like databases, you can create tables through the Query Browser GUI or via SQL commands. To use the Query Browser to create a table, first in the Schemata tab select the database you want to add a table to (see Figure 1 for a review of the Schemata tab). If you’re following along with your keyboard, you want to create the “superheroes” database again, since we’re going to use it (instead of “mystars”) for this series of examples. Then right-click and select the “Create New Table” menu option from the context menu. The MySQL Table Editor dialog will display as shown in Figure 3.

Figure 3 . The MySQL Query Browser’s Table Editor.


You’ll see the name of the database you’re working with displayed in the Database combo box. Enter the table name in the “Table Name” text box. The Apply Changes button is enabled as soon as you start typing in the Table Name text box. I’ll address columns shortly. But first, while we’re still working on the table, let’s look at other table-related options. Click on the “Table Options” tab to display additional table-related options as shown in Figure 4.

Figure 4 . Table related options in the MySQL Query Browser. The Storage Engine option button allows you to choose which storage engine to use for the table being created. MySQL will, by default, use the storage engine choice configured in the my.ini/my.cnf file. If, for some reason, you want to use a different engine for this table, you can use the Storage Engine option button to explicitly choose which engine. You might also want to set up a single database (and all of its tables) with a different engine than the configuration file’s default. You may be wondering why you wouldn’t just change the default engine, create your database, and then switch the default engine back? Because the change to the configuration file would affect all users, and if another user was creating a database after you flipped the switch, they would end up inadvertently creating their database with the wrong storage engine type. It’s much better to be able to use the flag to override the default for a one-off database need. You can change the Storage Engine type either before you begin specifying columns for your table, or at the very end, just before you click the “Apply Changes” button. I would recommend you change the type right away; once you get into the heat of the column definition operation, you’re liable to forget to go back to the Table Options tab.

194


The character set options in the bottom of the Table Options tab allow you to change the character set and collation sequence; the typical reader of this book probably won’t use either of those. You can also use a SQL command to create a table in a database, like so: create table 'superheroes'.'heroes' () engine = MYISAM

or create table 'superheroes'.'heroes' () engine = INNODB

The “engine =” specification is only needed if you’re creating a table different from that specified in the configuration file. If you’re writing scripts to save and reuse on multiple systems, it’s a belt-and-suspenders Best Practice is to specify as many options as you can, to avoid problems with defaults not being what you’re expecting. I’ll cover the specifics of the column definition clauses shortly. The Advanced Options tab, shown in two pieces in Figures 5 and 6, has all sorts of goodies for advanced users whose environments have special requirements.

Figure 5 . The top half of the Advanced Options tab in the MySQL Query Browser.


Figure 6 . The bottom half of the Advanced Options tab in the MySQL Query Browser. The prompts next to each option describe the options pretty well; the typical reader will most likely not use any of them. However, it may be interesting to some readers that the autoincrement value also displays the next value to use for the field identified as autoincrementing. Now it’s time to get to the fields.

Field attributes Just as a database without tables isn’t very interesting, a table without fields is pretty darn dull as well. In fact, while you can have a database without tables, you have to specify at least one field when creating a table. In the Query Browser, you define columns in the Columns and Indices tab, shown in Figure 7. You can either enter a row in the grid in the top half of the tab, or in the Column Details tab on the bottom half. In Figure 7, the primary key field, iidhero, is defined.

196


Figure 7 . The Column Details tab of the Query Browser’s Table Editor. If you’re using the grid, double-click in the text box under the Column Name header and enter the name of the field. Specify the datatype for the column, whether it can hold null values, if it should be the auto-incrementing field, check the appropriate flags (the flags that display vary according to the datatype), and specify a default value if desired. If you’re using the Column Details tab, you first need to use the grid to create a column name. As soon as you exit the Column Name text box in the grid, the Column Details controls are enabled and the Name is filled with the name of the field you just created. You can then use the rest of the controls in the Column Details tab to specify attributes of the new column. Once you define at least one column, you can click the “Apply Changes” button. The first time you do so while creating a table, MySQL will issue a ‘create table’ command behind the scenes. After you create a table, making changes to the column definitions (either adding new fields, or modifying or deleting existing fields), will cause MySQL to issue an ‘alter table’ command. When typing a datatype, you’ll find that, no, there isn’t a combo box pre-populated with allowable choices. However, the text box will auto-complete after you type enough distinct characters, which will save you from trying to define a field with a datatype of “INTEGRE.” Whether you choose to specify a field as ‘not null’ or not is up to you, depending on your specific situation. If you indicate a field cannot contain Null, though, you need to provide a default value, which will prevent table inserts from failing if you forget to manually provide a value for a ‘not null’ field. The auto-increment property can only be set for one field in a table, and is typically used for the primary key. The flags that are available vary according to the datatype. For example, “ZEROFILL” is available for integers, but not characters.


Note that the Query Browser will let you specify inappropriate choices, such as a primary key field defined with a VARCHAR data type; you get an error message after you click the “Apply Changes” button.

Figure 8 . A complete table definition for heroes. As with the Query Browser, you can create an invalid CREATE TABLE statement, such as specifying more than one column as auto-increment, or defining a column as autoincrement but not setting it as a key. However, just because you can type the command doesn’t mean it’ll work – MySQL will reject it when you try to execute it. It can be frustrating to create a long, detailed table definition, only to have the engine reject your structure for some subtle reason or unknown requirement. As you start to create tables in MySQL, I suggest that you create your table with the basic fields – a primary key, time stamps if you’re going to use them, and the first couple of important data fields. Once the table is up and running, add fields a few at a time, and if a particular modification fails, you only have to examine the last few changes to determine where you went wrong. You can also create tables via SQL commands. The following command performs the same function as the table definition shown in Figure 8. create table superheroes.hero ( iidhero integer unsigned not null auto_increment, cthe char(5) not null default '', cna char(50) not null default '', csecretidentityf char(20) not null default '', csecretidentityl char(20) not null default '', primary key(iidhero) )

198


engine = MYISAM

Indexes The Indices tab at the bottom of the Columns and Indices tab (shown earlier in Figure 3) allows you to specify indexes for the table. You must have an index on the primary key of the table, and that index is created for you automatically. You use this tab to create additional indexes. You need to create indexes on all keys in the table, both primary and foreign. You also need indexes on fields that will be searched upon frequently. Deciding how often “frequently” is will depend on your application; it’s more of an art than a science. You may make changes on indexes after your application is used for a while. By the way, you are probably aware of VFP’s SYS(3054) function that displays optimization information for SQL queries (known as “SQL ShowPlan”). MySQL has something similar: the EXPLAIN command. It describes which indices a query will use, so you can determine the fields an index should be used on. However, you should stay away from indexes on certain fields, such as fields with very few values (like TINYINT fields used for logicals) or complex fields (like blobs and large varchar fields). You can create indexes on these fields (with blobs and text types, you need to specify how many characters in the field you want the index to include, much like indexing a memo field on “left,25”). However, like with VFP, the more indexes you have, the slower other operations (like inserts) become. And an index on a field that only has three unique values isn’t going to do much good, will it? In order to create an index in the Query Browser, select the Indices tab in the Table Editor and click on the (+) symbol below the list box on the left side. The Add Index dialog will display as shown in Figure 9.

Figure 9 . Being prompted for an index name. MySQL provides a default index name, such as Index_2, which isn’t very descriptive, and thus not very helpful. After you change the name (if you wish), the index is displayed in the list box, ready for you to describe. You can create an index expression in one of two ways. The first way is to select a column in the grid, and then click the (+) symbol to the right of the Index Columns box in the lower right. Doing so displays the column in the Index Columns box. You can create a multiple field index expression by repeating this process. You can also highlight a field in the Index Columns box and click the (-) symbol to remove it from the expression. If you need to use just the first N characters of a field to include in the index expression, highlight the field in the Index Columns box, click the (>) symbol, and then enter the value N in the resulting text box. The field expression will change to something like this


csecretidentityl(6)

where N was the value 6. The second method is to use drag and drop. Highlight a column in the grid in the top of the tab, and then simply drag it into the Index Columns box in the lower right to create the index expression. You can drag multiple fields (one at a time) into the Index Columns box to create a compound index expression, and you can modify a specific field to use just the first N characters with the (>) symbol as well. The result of creating an index on last and first name is shown in Figure 10.

Figure 10 . The Table Editor with a newly created index. Notice that MySQL automatically created an index on the primary key. You can do this programmatically, with a command like so: ALTER TABLE 'superheroes'.'hero' add index 'csecretidentitylf'('csecretidentityl', 'csecretidentityf')

Creating primary keys It’s easy to create an auto-incrementing primary key. In the Query Browser, the settings to create a primary key that automatically updates upon record inserts (for a new ‘villain’ table) are: column name: iidvillain datatype: integer autoinc: checked

200


And select the “Primary Key” checkbox in the Column Details tab. Or during a create table statement: CREATE TABLE `superheroes`.`villain` ( ìidvillain` INTEGER UNSIGNED NOT NULL AUTO_INCREMENT, PRIMARY KEY(ìidvillain`) )

Once you add the auto-increment field, you can ignore the column during inserts: insert into villain (cnaf, cnal) values ("The", "Penguin")

produces iidvillain 37

cnaf The

cnal Penguin

Assigning foreign keys The Foreign Keys tab at the bottom of the Columns and Indices tab, shown in Figure 11, allows you to relate a foreign key in this table to the primary key in another table. Doing so then allows you to create relational rules and constraint actions between the two tables. (Note that the foreign key actions only work if your tables are created with the InnoDB storage engine.)

Figure 11. 11. The Foreign Keys tab in the bottom of the Columns and Indices tab of the Query Browser Table Editor.


Suppose you want to create a foreign key in the Sidekick table that relates to the Hero table. (Each sidekick must be attached to a hero, or else they get kicked out of Superhero Academy.) First, add a column to the Sidekick table, iidhero, that points to the Hero’s primary key, iidhero, as shown in Figure 12.

Figure 12 . The Foreign Keys tab of the MySQL Table Editor. Now, identify it as a foreign key. Click on the Foreign Keys tab in the bottom of the Columns and Indices tab, and click the (+) symbol underneath the list box on the left. The Add Foreign Key dialog displays, as shown in Figure 13.

Figure 13 . Being prompted for a name of a new foreign key. You’ll see that the foreign key name proposed by MySQL has nothing to do with the name of the field that is the foreign key. MySQL just uses such names internally to keep track of the foreign keys. Rename it if desired and click OK. The Foreign Key displays in the list

202


box, ready to be defined, with the foreign key name displayed in the Foreign Key Settings area, as shown in Figure 14.

Figure 14 . Getting ready to define the attributes of a new foreign key. First, identify the parent table in the Ref. Table combo box – in this case, Hero is the parent. MySQL attempts to determine the column in the current table that represents the foreign key and the reference column in the parent table (the primary key) that maps to the foreign key in the current table. The guess that MySQL makes is displayed below the Ref. Table combo box. In unambiguous cases, those guesses will be correct. If your data structure is complex enough (say, the foreign key isn’t named the same as the primary key in the parent table), you need to specify the column and reference column. If MySQL already made a guess, you can delete the values by right-clicking on the values under the Column or Reference Column headers and select the “Remove Column” menu choice. Then, double-click in the empty cell below the Column header to display the available choices for the foreign key in the current table, as shown in Figure 15.


Figure 15 . Setting up a foreign key manually. Do the same in the cell under the Reference Column header; the values in that combo box will be the available fields in the table identified in the “Ref. Table” combo box above. This way, you can pair a foreign key in the current table with a primary key in another table even if they’re not the same name. Finally, select the On Delete and On Update actions as needed, as shown in Figure 16.

Figure 16 . A completed foreign key.

204


Your choices for the actions are No Action, Cascade, Set Null, or Restrict for both actions. Here’s what will happen for each choice:

On Delete actions These choices control what happens when you delete a row in the parent table that has one or more children in the current table. For example, suppose the Hero table has a row for Batman, and the Sidekick table has rows for Robin and Batgirl. The On Delete action determines what happens to the Robin and Batgirl rows if you delete the Batman row. No Action: Nothing happens when you delete a row in the parent table. Your child row is orphaned. Robin and Batgirl are no longer attached to a superhero. The values for the foreign key will be meaningless, as the primary key won’t exist anymore. Cascade: All child rows are deleted when you delete the parent row. Relational integrity is maintained. Robin and Batgirl are automatically deleted once Batman is deleted. Set Null: Deleting the parent row sets the foreign key values in the child records to null. While Robin and Batgirl are still orphaned, it will be easier to find them if you want to reattach them to a new superhero. (If you leave a foreign key value in an orphaned record, the SQL SELECT command to find those orphaned records involves a ‘not in’ subclause, while records with a null foreign key simply need a ‘where isnull’ type of construct.) Restrict: Parent records with children cannot be deleted. The Robin and Batgirl records will need to be deleted or assigned to another superhero record before Batman can be deleted.

On Update actions These choices control what happens when you change the key of the parent that the foreign key of the child points to. For example, suppose the primary key value (iidhero) of Batman was 63. Then the Robin and Batgirl values for iidhero would also be 63. On Update rules control what happens when Batman’s 63 is changed to 262. No Action: Changing the key of the parent won’t affect the child at all. Batman’s iidhero gets changed to 262, but the Robin and Batgirl iidhero values of 63 stay at 63. They are no longer sidekicks of Batman – they become orphan sidekicks. Cascade: Changing the key of the parent automatically changes the foreign key of the child. Changing Batman’s iidhero from 63 to 262 means Robin and Batgirls’s iidhero values are also changed from 63 to 262. Set Null: Changing the key of the parent forces the foreign key of the child to be set to null. Robin and Batgirl’s iidhero values are set to null, and thus they are no longer sidekicks of Batman – like with No Action, they become orphan sidekicks. (This should probably be the default action, actually, as most sidekicks start out their career as orphans, huh?) Restrict: The parent’s key can’t be changed if there are children related to the parent. If Batman has one or more sidekicks, his key can’t be changed without first assigning his sidekicks to another superhero. Like everything else, you can also perform foreign key changes in code. For example, the command ALTER TABLE 'superheroes'.'sidekick' add constraint 'FK_iidhero' foreign key 'fk_iidhero' ('iidhero') references 'hero' ('iidhero') on delete CASCADE on update CASCADE


will add the foreign key just described in the GUI.

Working with time stamps There are a couple of subtleties when working with time stamps because of the restrictions that MySQL has. The most important restriction is that you can only have one auto-updating time stamp in a table. Another, more subtle, restriction is that, u nless explicitly defined otherwise, the first timestamp field in a table is the one that is automatically updated. Given these restrictions, let’s examine how to work with time stamps. ALTER TABLE `superheroes`.`sidekick` ADD COLUMN `tadded` TIMESTAMP DEFAULT NOW() ON UPDATE NOW() AFTER ìidhero`;

works fine – when you insert into sidekick (cna, csecretidentityf, csecretidentityl) values ('Batgirl', 'Barbara', 'Gordon')

you get the current date/time in the timestamp field. However, if you add a tchanged field that is supposed to be updated, like so: ALTER TABLE `superheroes`.`sidekick` ADD COLUMN `tchanged` TIMESTAMP DEFAULT NOW() ON UPDATE NOW() AFTER `tadded`;

this does not work. Instead, you get an error message: Incorrect table definition; there can be only one TIMESTAMP column with CURRENT_TIMESTAMP in DEFAULT or ON UPDATE clause

So, it seems that you have a choice to make. If you want the tadded field to be hit upon creation of the record, you have to hit the tchanged field yourself. Else, if you want the tchanged field to be hit upon any modification to the table, you have to do the original hit to tadded yourself. Since tadded only gets modified once, upon initial creation of the record, it makes sense to have it taken care of during the INSERT command, rather than via a default rule. Thus, tadded doesn’t have to be a timestamp field. Tchanged, on the other hand, gets updated all of the time, which makes it the perfect candidate for the sole timestamp field in the table. ALTER TABLE 'superheroes'.'sidekick' ADD COLUMN 'tadded' DATETIME DEFAULT 0, ADD COLUMN 'tchanged' TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP

Now add a brand new record: insert into sidekick (cna, csecretidentityf, csecretidentityl, tadded) values ('Robin', 'Dick', 'Grayson', now())

This results in

206


iidsidekick 10

cna Robin

..... .....

tadded 2006-06-08 16:33:22

tchanged 2006-06-08 16:33:22

Note that the tadded value is added manually and the current_timestamp default value for tchanged updates that field automatically. A moment later, the command update superheroes.sidekick set cna = "Boy Wonder" where iidsidekick = 10

generates this: iidsidekick 10

cna Boy Wonder

..... .....

tadded 2006-06-08 16:33:22

tchanged 2006-06-08 16:35:16

Note that the tchanged gets hit automatically (and the tadded stays the same, of course.) A tempting idea is to try current_user() for the cadded and cchanged fields default values to record who the user responsible for the add or update was. Unfortunately, at this writing, there’s a bug in MySQL that stuffs the string ‘current_user()’ into the field, not the value of ‘current_user()’. This bug has been logged (#17809) and is on the list to be included as soon as expressions are allowed in the DEFAULT clause.

End result: empty data set – what’s it look like? You’re finally done with the creation of your database. Hooray. You may be thinking we’re done, but au contraire! Let’s finish up by showing you how to see what your database structure looks like – sort of the equivalent of list structure

in Visual FoxPro. There are two methods you can use to get metadata information. The first is from the command line, via the “mysqlshow” command that we briefly touched upon during installation. The command “ mysqlshow” displays information about databases, tables, and columns – as long as you have some privileges to the item in question. The second is a pair of SQL commands you can use within the MySQL Monitor, or anywhere else you type in commands (like the Query Browser’s edit window.)

mysqlshow You can issue the “mysqlshow” command from an operating system prompt in either Windows or Linux; the results are the same. The simple mysqlshow command without any additional specifications looks like this: prompt> mysqlshow -u root -p

and produces results like this:


+------------------------+ | Databases | +------------------------+ | information_schema | | mysql | | mystars | | superheroes | | test | +------------------------+

You can add clauses to the “mysqlshow” command to drill down further. For example, you can add the name of a database to display the tables in that database, or the name of a database and a table in that database to display the fields in that table. prompt>mysqlshow -u root -p superheroes Enter password: ******** Database: superheroes +------------------------+ | Tables | +------------------------+ | hero | | sidekick | | villain | +------------------------+

or prompt>mysqlshow -u root -p superheroes sidekick Enter password: ******** Database: superheroes Table: sidekick +--------------------+--------------------+--------+-------+----------------+ | Field | Type | Null | Key | Default | +--------------------+--------------------+--------+-------+----------------+ | iidsidekick | int(10) unsigned | NO | PRI | | | cthe | char(5) | NO | | | | cna | char(50) | NO | | | | csecretidentityf | char(20) | NO | | | | csecretidentityl | char(20) | NO | | | | iidhero | int(10) unsigned | NO | MUL | 0 | +--------------------+--------------------+--------+-------+----------------+

(The actual list of attributes for a table is more detailed, but wouldn’t fit on a page. This display gives you an idea of some of the information available through “mysqlshow”.)

SQL commands You can also issue SQL commands that display various pieces of information about the database and associated structures. You can pass these commands as parameters in the MySQL monitor, in the Query Browser command window, or even pass them to MySQL from Visual FoxPro via SQL Pass-through. The “show databases” command, predictably, displays all databases you have at least some rights to, as shown in Figure 17.

208


Figure 17 . Using “show databases” in the Query Browser. Similarly, the “show tables” command lists all the tables in the currently selected d atabase (as indicated by the name in bold in the Schemata tab on the right side). See Figure 18.


Figure 18 . The “show tables” command displays all tables for the currently selected database. Finally, the “describe” command gives you detailed information about a specific table. Separate the name of the database and the table with a period, as shown in Figure 19.

Figure 19 . The “describe” command gives you detailed information about a specific table.

210


Getting data into the database There are several ways to populate an empty database. I cover them in the next chapter, “Populating a MySQL Database.”

Conclusion/Summary Visual FoxPro’s available data types keep growing with each major version. With version 9, VFP can match MySQL nearly one-to-one, so it’s often straightforward to map a VFP table’s structure to a MySQL structure. Using a GUI tool like the Query Browser, or getting up to speed with the SQL commands CREATE TABLE and ALTER TABLE, allows you to set up your database structure in short order as well. Now it’s on to the data! Updates and corrections to this chapter can be found on Hentzenwerke’s Web site, www.hentzenwerke.com. Click “Catalog” and navigate to the page for this book.

Chapter 11: Populating a MySQL Database: LOAD DATA INFILE 211

Chapter 11 Populating a MySQL Database: LOAD DATA INFILE Just as a database without tables or a table without fields isn’t very interesting, a database without data isn’t likely to attract much attention. In this chapter, I cover the first of two primary mechanisms for getting large amounts of data into your new MySQL database: using the native MySQL LOAD command. In the next chapter, I’ll cover writing a conversion program that spins through a DBF and populates the corresponding MySQL table via SQL INSERTs.

For the Visual FoxPro developer, there are several different mechanisms to load a MySQL database with data. The first is to type the data. Avenues include through a GUI tool like the Query Browser, or via manual INSERT INTO statements via the MySQL Monitor. Great if you just need to populate a lookup table with a half dozen rows; not so great if you need to load a multinational company’s product catalog. The second is to use a third party conversion tool. Note that the MySQL Migration Toolkit does not currently support migration from VFP – only Access, SQL Server, and Oracle. However, xCASE, the database design tool from RESolution Ltd. (http://www.xcase.com) has a mechanism to convert DBFs to MySQL. I don’t cover this process other than simply mentioning it because you obviously need a copy of xCASE to take advantage of the capability, and at $399, it’s unfortunately not a tool that a lot of VFP developers own. Another tool that has won plaudits from reviewers is “Full Convert” from Spectral Core (http://www.spectralcore.com/fullconvert/index.php). The third mechanism uses the native MySQL LOAD functionality built into MySQL. Since LOAD is designed for a broad range of users who have varying needs, it isn’t tuned for the specific needs of DBF users. However, it can be useful in some cases and the typical VFP/MySQL developer should be aware of and minimally conversant with it. The purpose of this chapter is to cover LOAD in some detail. The fourth is to write a program to spin through an existing set of tables and insert that data into corresponding tables in the new MySQL database. I’m going to cover this process in detail in the next chapter. As you’ll see in future chapters, I’m a big fan of SQL Pass-Through, so that’s the mechanism I’ll use with this program. Data conversion can be frustrating because it’s filled with trial and error, and frequently gets short shrift from the developer’s attention. And even if the developer has allotted enough time, management often dismisses the complexities. (“I know you had planned for two weeks to do the migration, but how hard can it be? It’ll probably only take a couple of days!”) However, chances are that you’ll run a migration more than once, particularly in a RAD environment where the target data structures repeatedly change. As a result, I’ll spend a fair amount of time discussing how to do it and some of the pitfalls you may encounter along the way.

212


The MySQL LOAD DATA INFILE command The LOAD DATA INFILE command reads at a very high speed rows from a text file into a table. This is a great tool for moving clean data with a known good format into a MySQL table. Let’s use our “mystars” database from Chapters 8 and 10 again. Suppose you’re working on a Windows workstation that has a text file named “notastar.txt” (say it slowly: “Not a star”) in the root of Drive E. This text file looks like this: 1, 2, 3, 4, 5,

al bob carl donna edgar

This is an ASCII text file created in a pure text editor (not WordPad or, heaven forbid, Microsoft Word.) Examples of appropriate text editors are Microsoft Notepad, a third party editor like Notepad++ (notepad-plus.sourceforge.net), or even Visual FoxPro. Note that with these, each line will end with a CRLF combination: chr(13), chr(10). We’ll deal with those characters in a minute. Also suppose you have a table, ‘notastar’, in the ‘mystars’ database with the following structure: iidnotastar cnaf

integer varchar(45)

The command LOAD DATA INFILE ‘e:/notastar.txt’ INTO TABLE mystars.notastar FIELDS TERMINATED BY ‘,’

will create a new record for each row in the text file. Let’s take this command apart, piece by piece.

Where do I run the command? The first question that comes to mind is “where do I type this command?,” followed shortly by “where is the MySQL server and the database?,” both very good questions. The answer is that you can type the command into any place you would issue any other MySQL SQL command, such as the Query Browser or the MySQL monitor. When you log in to the QB, for example, you’ll need to identify which MySQL server you’re attaching to, and, optionally, which database you want to use. (Otherwise you’ll have to pick a database once you’re in the QB.) So in this scenario, I’m sitting in front of a Windows workstation. Everything is on this workstation – the ‘mystars’ database, the ‘notastar.txt’ text file, the MySQL server, and the Query Browser executable. I logged into the local MySQL server via the Query Browser, told it to attach to the local ‘mystars’ database, and then loaded the ‘notastar.txt’ file sitting on drive E.


What if I wanted to attach to a MySQL server on another box, say, on a box in a hardened bunker 5 miles below the Rocky Mountains? I could do that just as easily, as long as I had permission, of course. However, dealing with the text file is different. MySQL assumes the text file is on the same box (‘host’) as the MySQL server, so the above LOAD DATA INFILE command would assume the MySQL box sitting under the Rocky Mountains has a drive E, and that there’s a ‘notastar.txt’ file on that drive. What if you’re connected to the Rocky Mountain MySQL server, but the text file you want to load is sitting on your local machine? You need to include the “LOCAL” clause in the command, like so: LOAD DATA LOCAL INFILE ‘e:/notastar.txt’ INTO TABLE mystars.notastar FIELDS TERMINATED BY ‘,’

Actually, the text file doesn’t have to be physically on YOUR local machine, it just needs to be accessible from the local machine; it could be on a network share, like so: LOAD DATA LOCAL INFILE ‘n:/somedirectory/notastar.txt’ INTO TABLE mystars.notastar FIELDS TERMINATED BY ‘,’

If I was working on my Linux workstation right now, the account I was using would need rights to the text file if it wasn’t in my home directory. And if the text file was on another machine, I’d need rights to that location as well. Now let’s look at the syntax of the command.

LOAD INFILE syntax As you can see, the comma is the separating character between fields, and is explicitly identified in the LOAD command. If you had fields that contained commas as part of the data, such as addresses (321 Rue Morgue, #100), then you need to use a different separation character. If you’re not watching carefully, you may miss the fact that the leading spaces before each name are considered part of the string being brought in, so the ‘cnaf’ field contains ‘ al’, not ‘al’. In other words, the LOAD command doesn’t do any type of trimming when importing. Furthermore, this command will also bring the chr(13) + chr(10) character combination at the end of each name, resulting in a backwards paragraph symbol appearing in the Query Browser. You could either strip them from your database once you’re done loading the text file, or you could reformat your text file, such as replacing the chr(13) + chr(10) combination with a unique row termination character – one that won’t appear anywhere else in your data. For example, you could change the next few rows of your text file like so: 6,fauna|7,gloria|8,henrietta|9,ilsa|10,jennifer|

and then LOAD DATA INFILE ‘e:/notastar.txt’ INTO TABLE mystars.notastar FIELDS TERMINATED BY ‘,’ LINES TERMINATED by ‘|’

214


and Fauna, Gloria, Henrietta, Ilsa, and Jennifer will not be followed by those goofy backward Ps. OK, that’s it for a simple example. Let’s add some complexity.

Handling primary keys What about primary keys? In the example we just looked at, each row had a unique ID, but the table wasn’t explicitly defined as “iidnotastar” being a primary key. If the “iidnotastar” field was defined as a primary key, the previous two LOAD DATA commands would still have worked, provided there weren’t already rows in the “notastar” table with key values that matched the values in the text file. In other words, the LOAD command will accept text file rows with unique primary key values – but it’s up to you to make sure your text file contains only valid values. Once the LOAD command encounters a row with an invalid value, it stops processing. Annoying if the second row in a 466,900 row text file has bad data, huh?

Leading spaces We’ve now seen a simple integer and a simple character string get loaded into a table without any fuss. You can import numbers as character strings simply by defining the target field in the table as a character (or varchar) field. Note, of course, that leading spaces in front of the numbers in the string are included as part of the string being imported. In other words, suppose your table was structured like this: iidnotastar cnaf cvalue

integer varchar(45) char(10)

a text file like so: 11,Karl,200|12,Louie, 300|13,Morgan,400|

will be imported with a leading space before the field containing ‘300’. The string ‘ 300’ is contained in the ‘cvalue’ field for Louie.

Date and date-time fields Dates can be more troublesome, of course, but don’t have to be. Suppose you have a field in your table, dob, defined simply as “DATE,” Not Null, and with a default value of ‘0000-0000’. In your text file, use the format YYYY-MM-DD, without surrounding the date in quotes of any sort, like so: 19,Nolan,42,1980-10-1|

and the import will bring the date in as expected. However, you get an error if you surround your date with single or double quotes, like so: 20,Olga,,’2001-10-20’|21,Petravich,,”2001-10-21”|


You can use two digit years, subject to the rules in Chapter 10 about how MySQL will assume the century. (A date of 80-1-1 will be brought in as 1980-1-1, but a date of 45-5-8 will be brought in as 2045-5-8.) If your table’s field is defined as “datetime” instead of just “date,” you can include the time portion in the text file, like so: 22,Quint,,1956-12-05 12:20:27|

If you don’t include the time portion of a date-time field, the time is set to midnight (00:00:00). However, if you try to out-clever the MySQL engine by skipping the date and just providing a time, your time is converted to a date and inserted, like so: 23,Rutherford,,9:09:09|

produces a field containing 0009-09-09 00:00:00

probably not what you were expecting. If your time can’t be converted to a legal date, like this time just before 10 am: 24,Stollenwerk,,9:52:37|

the LOAD will fail with an error (“Incorrect datetime value: ‘ 9:52:37’ for column ‘dob’ at row 1”).

Handling BLOBS and TEXT fields If you’re importing a text file, you likely aren’t stuffing huge amounts of data into the MySQL fields that handle large field sizes like TEXT or BLOB. Instead, you’ll use a different technique. For example, suppose you have a directory full of promotional photos (.JPG files), one for each individual who is Not A Star. You create a new field in the ‘notastar’ table called ‘bpromophoto’. Now you have to stuff the binary data itself into the BLOB field for each record. Assuming you know which file goes into which record, the command would look like this: update mystars.notastar set bpromophoto = LOAD_FILE(‘e:/alvin_the_chipmunk.jpg’) where iidnotastar = 1

If you have a lot of files, however, this is somewhat of a pain, and you’ll need to write a program that spins through the directory contents and executes the SQL update command for each file in the directory. If the file does not exist or another situation causes the LOAD_FILE function to fail, LOAD_FILE will return null. In the Query Browser, the contents of a BLOB field are not shown, just like memo fields in an xBASE browse. Figure 1 shows two records in the Query Browser.

216


Figure 1. A BLOB field displayed in the Query Browser. The first record has data in the BLOB field, indicated by the word “BLOB” in the legend, while the second record’s BLOB field is empty. The two icons in the first record’s BLOB field cell provide access to two handy little tools. The magnifying glass allows you to view the contents of the BLOB field in binary format, as shown in Figure 2.

Figure 2 . The Query Browser Field Viewer for BLOB fields. If the BLOB field is, say, a recognizable image, the Field Viewer will have more than one tab, as shown in Figure 3.


Figure 3 . The Field Viewer displaying an image. And the Binary tab still shows the binary code for the file, as shown in Figure 4.

Figure 4 . The Query Browser Field Viewer for BLOB fields. The diskette icon in the BLOB field within the Query Browser allows you to copy the contents of the BLOB field to a file on disk; clicking on it simply displays a “File Save” dialog and prompts you for a file name (and allows you to navigate to a different place on disk if you wish). So that’s the skinny on the LOAD DATA INFILE command. There are a number of additional options; indeed, the on-line help topic is rather lengthy, describing all of the options and the nuances of each. (Search on “LOAD DATA INFILE” for more information). Very useful for importing clean text files that are already properly formatted. In the next chapter, we look at writing your own VFP program to move data from a VFP database into a MySQL database.

218


Conclusion/Summary Transferring your VFP data into your MySQL database is an important part of moving to MySQL as a backend for your VFP applications. In this chapter, we looked at the first of the two most common mechanisms for doing so. In the next chapter, we’ll look at the other mechanism and discuss some performance issues. Updates and corrections to this chapter can be found on Hentzenwerke’s Web site, www.hentzenwerke.com. Click “Catalog” and navigate to the page for this book.

Chapter 12: Populating a MySQL Database Programmatically 219

Chapter 12 Populating a MySQL Database Programmatically In this chapter, I cover the second of two primary mechanisms for getting large amounts of data into your new MySQL database: writing a conversion program that spins through a DBF and populates the corresponding MySQL table via SQL INSERTs.

While LOAD DATA INFILE is very useful for importing clean text files that are already properly formatted, oftentimes you’ll need to tinker with the data during the import. Let’s look at writing your own VFP program to move data from a VFP database into a MySQL database. This chapter centers on writing a program to spin through an existing set of tables and insert that data into corresponding tables in the new MySQL database. As you’ll see in future chapters, I’m a big fan of SQL Pass-Through, so that’s the mechanism I’ll use with this program. Data conversion can be frustrating because it’s filled with trial and error, and frequently gets short shrift from the developer’s attention. (“We’re running late on the system. We had planned for two weeks to do the migration, but it’ll probably only take a couple of days, so that’s where we’ll cut the schedule to make up time!”) However, chances are that you’ll run a migration more than once, particularly in a RAD environment where the target data structures repeatedly change. As a result, I’ll spend a fair amount of time discussing how to do it and some of the pitfalls you may encounter along the way.

Converting DBF data to MySQL Data migration is much like deployment, in that it’s assumed to be a one-off, almost trivial aspect of the project. And as anyone who has done a deployment, those are invariably invalid assumptions. Data migration (like deployment) is typically done during one phase of a project, albeit usually multiple times, and thus it’s not a process developers get as much experience with like other parts of the application. Additionally, many people feel that moving their data from one storage engine to another is going to be a one-time shot, (again, just like deployment!). As a result, they often don’t address the process with the same amount of rigor as they do with the rest of their application. Documentation is less detailed, if it exists at all, and any type of commenting (not to even mention up-to-date commenting!) in the data conversion routines is even rarer than it is in the application itself. And not having regular exposure to the code means, when it comes time to make changes (and that time will come!), documentation will be that much more important. The fact that migration is done less frequently, however, is all the more reason to approach the process with rigor and discipline. Rare is the environment when a conversion goes smoothly, and perfectly, the first time. One of the fundamental rules of application development is that customers never know their data – no matter what they tell you, you have

220 MySQL Client-Server Applications with Visual FoxPro

to perform data validation, boundary case testing, and error-trapping, or else you risk letting garbage get into your new database.

An overview of the process The process of migrating data in a VFP DBF to its partner in a MySQL database table involves the following steps: 1. Open the DBF 2. Connect to the MySQL database 3. Spin through the DBF records 4. For each DBF record, perform a SQL INSERT into the MySQL table 5. Summarize the results If your database is more complex than one table, though, this process will need to be enhanced somewhat. Thus, I’m going to perform several conversions in this section, each with increasing complexity. (Sample source code, data files, and instructions for using said sample source and data are provided at the end of each example.) The first will be very simple. Just one table to one table, with each table having only a couple of fields, and virtually no error trapping, so you can see the entire process without having to wade through pages and pages of extraneous detail and distractions. Once you’ve seen the basic steps, we’ll do a second conversion, with error handling included, so you can see what you might do to trap and handle the inevitable problems that occur in the real world. The third conversion will explore the issues involved with a wider variety of data types, multiple tables, and fields that don’t match up one-to-one between databases. Finally, I’ll offer some ideas on how to make the process reusable.

Setting up the MySQL tables for this chapter We created the SuperHeroes database in chapter 10 and the MyStars database in Chapter 11. For our purposes in chapter 12, we’re going to create a new database. Why? One reason is, as the answer to the old joke “How do you get to Carnegie Hall?” goes, “practice.” The more times you create and modify databases and tables in MySQL, the better off you’ll be. Another reason is that the first two databases were sort of, er, whimsical. This database is going to be more grounded in real-world applications.

Description of the sample database The sample database for this chapter is essentially a name and address schema for businesses as shown in Figure 1.


Figure 1. The schema for the INS database. BIZ - Business: Each record represents a single business. PERSON - Person: Each record contains one individual associated with the business. One or more individuals can be associated with a single business. LOC - Location: Each record represents a single physical location associated with a business. The location table has an “HQ only” field for records that represent headquarters information. COOR - Coordinate: Each record represents a single point of contact (phone number, email address, etc.) for a location. A point of contact can be a voice number, fax number, email address, or web address. The lookup table, ZLOOKUP, contains a list of business categories that a business may fall into such as grocery, jeweler, dentist, and so on. Finally, the join table, BIZCAT, is a many-to-many table that relates BIZ and ZLOOKUP. It contains the keys for businesses and categories, as one business may be classified as more than one business type. For example, a pizza place may belong to “specialty foods,” “restaurant,” and “pizza parlor.” For simplicity’s sake (this isn’t a book on database design!), the tables in VFP and MySQL map one-to-one – there are biz, person, loc, coor, bizcat, and zlookup tables in both databases. However, in order to show how to massage the data during the conversion, the table structures of the VFP and MySQL ‘loc’ tables differ; the VFP table has a single ‘street address’ field while the components of the street address (number, direction, etc.) are broken up into separate fields in the MySQL table. Here are the structures for all of the tables. (All tables have cadded, tadded, cchanged, and tchanged fields; they’re not listed for readability.) • •

•

•


BIZ.DBF 1 IIDBIZ 2 CNABIZ 3 CNASEC PERSON.DBF 1 IIDPERSON 2 IIDBIZ 3 CPREF 4 CNAF 5 CNAM 6 CNAL 7 CSUF 8 MBIO LOC.DBF (VFP) 1 IIDLOC 2 IIDBIZ 3 LISHQ 4 CSTREET 5 CCITY 6 CSTATE 7 CZIP LOC.DBF (MySQL) 1 IIDLOC 2 IIDBIZ 3 LISHQ 4 CNO 5 EDIR 6 CSTREET 7 CSUF 8 CSECLINE 9 CCITY 10 CSTATE 11 CZIP COOR.DBF 1 IIDCOOR 2 IIDLOC 3 CDATA 4 CTYPE BIZCAT.DBF 1 IIDBIZCAT 2 IIDBIZ 3 IIDCAT ZLOOKUP.DBF 1 IIDZLOOKUP 2 CABLUP 3 CNALUP 4 CDE 5 ICD 6 CCDLUP

I C C

4 50 50

I I C C C C C M

4 4 5 20 20 20 10 4

I I L C C C C

4 4 1 50 35 10 10

I I L C ENUM C C C C C C

4 4 1 6 35 10 35 35 10 10

I I C C

4 4 100 10

I I I

4 4 4

I C C C I C

4 25 25 50 4 25

Sample source code The source code for this chapter is contained in the file CH12.ZIP. Unzip this file into an empty directory and follow the instructions in the README.TXT file contained therein.


A simple program skeleton We’re going to start with a small program that simply copies the data in one small VFP table to the corresponding table in MySQL, to show you how it’s done. Then we’re going to expand on that program, repeating the process several times, while adding complexity and more features. The CH12A.PRG program included in the source code files is merely the skeleton that we’ll use for subsequent iterations. It attempts to set up the connection and closes it at the end of the program if it was established correctly at the outset. In the middle, there’s a place holder for where the interesting code will go. In the listings in this chapter, I won’t repeat this skeleton each time; the listing will replace the place holder in the skeleton program. The programs in the source code files include the entire file. The skeleton program looks like this: * * * *

ch12a.prg VFP 9 SP 1 shell for program that will create database and tables in MySQL for database conversion exercises

lparameters m.tcUN, m.tcPW if pcount() < 2 messagebox("You must pass the username & password as a parameter, like so:" ; + chr(10) + chr(13) + chr(10) + chr(13) ; + "do CH12A with 'bob','secret'") return endif m.liH=sqlstringconnect( ; + "DRIVER={MySQL ODBC 3.51 Driver};" ; + "SERVER=localhost;UID="+m.tcUN+";PWD="+m.tcPW) if m.liH < 1 messagebox("No connection; handle is:"+transform(m.liH)+":") return else messagebox("success") endif **** * * all the interesting code will go here * **** m.liResult = sqldisconnect(m.liH) debugout iif(m.liResult = 1, "Successful disconnect", "Unsuccessful disconnect") * eof

Note that you have to run the program by passing the username and password for the MySQL account you’re using. I do this for two reasons. First, I don’t want to hard code my own username and password, so I’d have to put in dummy values. The problem is that those dummy values wouldn’t work on my machine, so I’d have to hard code my own values while


testing, and then remember to change the programs to contain the dummy values before I sent out the source code. Yeah, I’d never make a mistake there, would I? You neither. And second, if I hard coded these values, some readers would blow by the instructions to change them, run the programs, and then ask me why the program isn’t connecting to THEIR MySQL server. So in the long and short runs, it’s easier to simply pass both values as parms. If it annoys you, you can change the programs yourself and hard code your username and password in the programs.

The SQLEXEC command You should recall from Chapter 6 that the SQLEXEC command allows you to send SQL commands to the back-end server. The syntax looks like this: m.liResult = sqlexec(m.liHandle, m.lcCommand, m.lcCursor)

where •

• • •

m.liResult is an integer; 0 indicating the command is still executing, positive indicating success, or negative indicating failure; m.liHandle is the connection handle to the server; m.lcCommand is a string containing the SQL command; and m.lcCursor is an optional parameter containing the name of the cursor the result set will be put into, if applicable.

If m.liResult is -1 for a command that should create a cursor, the cursor may not be created.

Formatting commands for SQLEXEC SQLEXEC takes as its second argument any SQL command, such as SELECT, INSERT, or CREATE TABLE. It can be complex, and thus, error-prone, to format a long SQL command so it gets sent to the back end engine properly. Here’s a primer on how to do so, using INSERT, since INSERT exemplifies some of the potential problems. The basic INSERT command we want to create and execute with SQLEXEC will look like this: insert into ins.biz ; (iidbiz, cnabiz, cnasec) ; values ; (1, 'Last National Bank', 'of Nowhere')

Let’s take this apart.

Converting to a string for SQLEXEC() In order to be passed as a parameter in the SQLEXEC() function, the statement needs to be converted to a string. In the olden days, we could use those durned quotes and line continuation characters to build a string, like so: =sqlexec(m.liHandle, ; "insert into ins.biz " ;


+ "(iidbiz, cnabiz, cnasec) " ; + "values " ; + "(1, 'Last National Bank', 'of Nowhere')" ; )

The INSERT statement was actually broken up into four text strings, just like it was shown initially, and the four were concatenated. This is just a simple INSERT command, but you can see that it’s already getting clumsy to handle. Imagine the horrors of keeping quotes, commas, parens, and variable names straight as it grows. The horror doesn’t lie just with INSERT, though. Here’s what a simple CREATE TABLE command would look like: =sqlexec(m.liHandle, ; "CREATE TABLE ins.biz ( " ; + "iidbiz int(10) unsigned NOT NULL auto_increment, " ; + "cnabiz char(50) NOT NULL default '', " ; + "cnasec char(50) NOT NULL default '', " ; + "ctype int(10) unsigned NOT NULL default '0', " ; + "cadded char(10) NOT NULL default '', " ; + "tadded datetime NOT NULL default '0000-00-00 00:00:00', " ; + "cchanged char(10) NOT NULL default '', " ; + "tchanged timestamp NOT NULL default CURRENT_TIMESTAMP " ; + "on update CURRENT_TIMESTAMP, " ; + "PRIMARY KEY (iidbiz) " ; + ") ENGINE=MyISAM DEFAULT CHARSET=latin1" ; )

Do you want to mess with these quotes and line continuation characters? The first step is to create a variable, m.lcStr, that will hold the SQL command, and then pass that variable to SQLEXEC(). m.lcStr ; = "insert into ins.biz " ; + "(iidbiz, cnabiz, cnasec) " ; + "values " ; + "(1, 'Last National Bank', 'of Nowhere')" ;

Which is passed as a parameter to SQLEXEC() like so: =sqlexec(m.liHandle, m.lcStr)

You’ll find this technique makes it a lot easier to debug a SQL command that has gone awry – for example, it’s easy to send m.lcStr to the Debug Output window to examine the string actually being passed to SQLEXEC, not the string you think is being passed! But this is still going to cause us grief past about two field names. Or maybe not even that far. Let’s take a look at the situation when we want to use variables instead of hard-coded values.

Using variables SQL INSERTS that rely on hard-coded values are easy to write as strings, but they’re not very useful in dynamic database applications. Instead of inserting hard-coded values, we need to substitute variables of one sort or another for the constants. You could use SCATTER


MEMVAR into variable names and use those variable names in the INSERT, or simply reference the fields in the table themselves. I’ll go with the latter since it’s one less set of steps. The plain INSERT looks like this now: insert into ins.biz ; (iidbiz, cnabiz, cnasec) ; values ; (biz.iidbiz, biz.cnabiz, biz.cnasec)

Converting this into a string looks like this: m.lcStr ; = "insert into ins.biz " ; + "(iidbiz, cnabiz, cnasec) " ; + "values " ; + "(" + allt(trans(biz.iidbiz)) + ",'"+ biz.cnabiz +"','"+ biz.cnasec +"')"

There’s a subtle difference in the way various fields are handled. In the example using hard-coded values, you saw that the hard-coded values are passed differently, according to what type of data they are: + "(1, 'Last National Bank', 'of Nowhere')"

The primary key in the DBF is an integer, so it doesn’t need any delimiters. However, it does need to be converted to a string in order to be added to the INSERT string being passed to SQLEXEC(). If you were going to pass just a single integral value, the code to do so would look like this: "(" + alltrim(trans(biz.iidbiz)) + ")"

Character strings, on the other hand, do need delimiters. However, since a string needs to be delimited, the code for passing a single character field requires beginning and end quotes, and would look like this: "('" + biz.cnabiz + "')"

The difference is the single quote inside the beginning and ending parens, which act as the delimiters for the text string. Later on, we’ll strengthen this code to cover strings that might include quote characters.

Handling multiple data types It gets trickier (well, messier, at least) when you have multiple fields that need to be separated by commas, with some fields needing delimiters and others not. The first example shows two character fields (note the single quotes around both fields), while the second shows an integer and a character field (with single quotes needed only around the second field): "('" + hero.cthe + "','" + hero.cna + "')" "(" + alltrim(trans(hero.iidhero)) + ",'" + hero.cna + "')"


Ugh! There has to be a better way. Enter the TEXT/ENDTEXT command!

Tip: Getting the structure of an existing MySQL table By the way, if you already created a table, you can use the “Copy SQL to Clipboard” command to capture the script needed for creating a table. Open the Query Browser and right click on a table in the Schemata tab on the right side of the Query Browser dialog. The “Copy SQL to Clipboard” command is the third item in the menu. After you select the menu option, switch to the application where you want to place the script, and select Edit | Paste or just hit Ctrl-V. The entire CREATE TABLE command will be placed into the window you selected. Note that doing so changes the appearance of the command a bit. MySQL surrounds table and field names with tick marks, like so: = "CREATE TABLE `xxx`.`biz` ( " ; + "ìidbiz` int(10) unsigned NOT NULL auto_increment,

" ;

I took the tick marks out of the previous listings because they add to the clutter, but they’re left in the code in the source code files. You’ll also want to note that the SQL that MySQL generates has a semi-colon at the end of each statement ( not each line), which is MySQL’s line termination character. If you forget to delete the semi-colon when you paste the SQL string into VFP, you will likely get unexpected results.

A better way to format SQL commands for SQLEXEC You may be thinking that having to surround the pieces of a SQL command with quotes as done in the previous listings would be a pain, and if so, you’re right. Thus, it’s an appropriate time to introduce a command that will make your life using SQLEXEC much, much easier. The TEXT/ENDTEXT commands in VFP 9 will convert a block of text to a string without having to mess with any of the messy beginning and ending quotes. Here’s an example. Old way, using quotes: m.lcStr = "CREATE TABLE `xxx`.`biz` ( " ; + "ìidbiz` int(10) unsigned NOT NULL auto_increment, + "PRIMARY KEY (iidbiz) " ; + ") ENGINE=MyISAM DEFAULT CHARSET=latin1"

" ;

New way, using TEXT/ENDTEXT: text to m.lcStr noshow CREATE TABLE `xxx`.`biz` ( ìidbiz` int(10) unsigned NOT NULL auto_increment, PRIMARY KEY (iidbiz) ) ENGINE=MyISAM DEFAULT CHARSET=latin1 endtext

The ‘noshow’ option suppresses the output of the command to the display, sort of like “noconsole” does with the REPORT and LABEL commands. If you haven’t used TEXT/ENDTEXT before, it can be disconcerting to expect multiple lines be part of a single


command, without the aid of a semi-colon. But once you get used to it, it’s a wonderful tool to have in your hip pocket. Let’s take a look at the INSERT command, because it’s going to introduce some complexities. If you simply try to surround the INSERT command with TEXT/ENDTEXT, you’ll get an undesirable result: text to m.lcStr noshow insert into ins.biz (iidbiz, cnabiz, cnasec) values (biz.iidbiz, biz.cnabiz, biz.cnasec) endtext

Unfortunately, m.lcStr will contain insert into ins.biz ; (iidbiz, cnabiz, cnasec) ; values ; (biz.iidbiz, biz.cnabiz, biz.cnasec)

and not insert into ins.biz (iidbiz, cnabiz, cnasec) values (1, 'Last National Bank', 'of Nowhere')

In other words, the values in the current record of BIZ aren’t sent to the back-end database. We need the variables, such as biz.cnabiz, incorporated into the string. We do this with textmerge delimiters, somewhat like we hard coded the quotes and commands earlier, but in a much easier fashion. Placing a VFP expression in between the “<<” and “>>” delimiters inside a TEXT/ENDTEXT expression, and including the additional option, ‘textmerge’, will cause the expression to be evaluated when the TEXT command is executed. Here’s an example: text to m.lcStr textmerge noshow insert into ins.biz (iidbiz, cnabiz, cnasec) values (<>, '<>', '<>') endtext

Executing this code results in the following value being assigned to m.lcStr: insert into ins.biz (iidbiz, cnabiz, cnasec) values (1, 'Last National Bank', 'of Nowhere')

You’ll see that the expression biz.cnabiz


is replaced with the value Last National Bank

However, since we need to send 'Last National Bank'

(with the single quotes), we include the single quotes outside the “<<” and “>>” delimiters in the TEXT command. If we wanted to trim the value of biz.cnabiz, we can include the ALLTRIM function inside the “<<” and “>>” delimiters like so: '<>'

Naturally, you can include VFP functions that return values, like date() or sys(2004) (which returns the VFP home directory.)

Creating the sample MySQL database The first interesting program we’ll write is the one that creates the MySQL databases. Not only will this help you avoid the tedium of having to manually create each table (and possibly introduce errors that would cause problems later), but we’ll learn our first trick in dealing with SQL PassThrough. Remember that the code displayed here is simply the ‘interesting’ part of the whole program.

Drop the existing tables The first thing in our “create MySQL tables” program is to get rid of the old tables, if they exist. The basic code looks like this: m.lcStr = "drop table if exists ins.biz" m.liResult = sqlexec(m.liH, m.lcStr) if m.liResult < 1 debugout "Command " + m.lcStr + " failed." endif

You’ll notice that I’m not using TEXT/ENDTEXT here, because it would actually be more work than just assigning the actual DROP command to a string. If the table exists, it’s dropped. If it doesn’t exist, MySQL ignores the drop command. If the SQLEXEC command returns a negative value, something went horribly wrong – a typographical error, perhaps. This is a little wordy, so we can shorten it a bit, like so: m.lcStr = "drop table if exists ins.biz" if sqlexec(m.liH, m.lcStr) < 1 debugout "Command " + m.lcStr + " failed." endif

or even more, like so: m.lcStr = "drop table if exists ins.biz"


m.lcStrErr = m.lcStrErr + iif(sqlexec(m.liH, m.lcStr) < 1, ; "Command '" + m.lcStr + "' failed. " + chr(13), "")

The full routine looks like this: m.lcStrErr = "" m.lcStr = "drop table if exists ins.biz" m.lcStrErr = m.lcStrErr + iif(sqlexec(m.liH, m.lcStr) < 1, ; "Command '" + m.lcStr + "' failed. " + chr(13), "") m.lcStr = "drop table if exists ins.loc" m.lcStrErr = m.lcStrErr + iif(sqlexec(m.liH, m.lcStr) < 1, ; "Command '" + m.lcStr + "' failed. " + chr(13), "") m.lcStr = "drop table if exists ins.coor" m.lcStrErr = m.lcStrErr + iif(sqlexec(m.liH, m.lcStr) < 1, ; "Command '" + m.lcStr + "' failed. " + chr(13), "") m.lcStr = "drop table if exists ins.zlookup" m.lcStrErr = m.lcStrErr + iif(sqlexec(m.liH, m.lcStr) < 1, ; "Command '" + m.lcStr + "' failed. " + chr(13), "") m.lcStr = "drop table if exists ins.bizcat" m.lcStrErr = m.lcStrErr + iif(sqlexec(m.liH, m.lcStr) < 1, ; "Command '" + m.lcStr + "' failed. " + chr(13), "") debugout m.lcStrErr

Yes, it’s a little redundant, but we’ll take care of that later in the chapter.

Create the tables Now it’s time to create the tables. The basic SQL ‘create table’ command looks like this: CREATE TABLE ins.biz ( iidbiz int(10) unsigned NOT NULL auto_increment, cnabiz char(50) NOT NULL default '', cnasec char(50) NOT NULL default '', cadded char(10) NOT NULL default '', tadded datetime NOT NULL default '0000-00-00 00:00:00', cchanged char(10) NOT NULL default '', tchanged timestamp NOT NULL default CURRENT_TIMESTAMP on update CURRENT_TIMESTAMP, PRIMARY KEY (iidbiz) ) ENGINE=MyISAM DEFAULT CHARSET=latin1

Wrapping it in TEXT/ENDTEXT results in this: text to m.lcStr noshow CREATE TABLE ins.biz ( iidbiz int(10) unsigned NOT NULL auto_increment, cnabiz char(50) NOT NULL default '', cnasec char(50) NOT NULL default '', cadded char(10) NOT NULL default '', tadded datetime NOT NULL default '0000-00-00 00:00:00', cchanged char(10) NOT NULL default '', tchanged timestamp NOT NULL default CURRENT_TIMESTAMP


on update CURRENT_TIMESTAMP, PRIMARY KEY (iidbiz) ) ENGINE=MyISAM DEFAULT CHARSET=latin1 endtext m.lcStrErr = m.lcStrErr + iif(sqlexec(m.liH, m.lcStr) < 1, ; "Command '" + m.lcStr + "' failed. " + chr(13), "")

No real need to repeat this for four more tables, would you say? The entire program is named CH12B.PRG.

A simple, one table migration Now it’s time to do the real work – copying the data over. The previous section covered how to create the MySQL tables, so I’m going to assume you now have them set up, hungry for data, and accessible by your day-to-day MySQL user. In the examples that follow, I removed the code that migrates the user and timestamps in order to increase readability. The complete code is included in the source code files.

Using SQL INSERT to copy data The core of this migration process is the SQL INSERT command, formatted as a string, and passed as a parameter in the SQLEXEC() function. use BIZ scan text to m.lcStr textmerge noshow insert into ins.biz (iidbiz, cnabiz, cnasec) values (<>, '<>', '<>') endtext m.lcStrErr = m.lcStrErr + iif(sqlexec(m.liH, m.lcStr) < 1, ; "Command '" + m.lcStr + "' failed. "+chr(13), "") endscan use in BIZ

The TEXT/ENDTEXT commands wrap around a single call to a SQL INSERT command, grabbing the values of the current record in the VFP version of the BIZ table, and stuffing them into the corresponding version of the table in MySQL. Note that the actual SQLEXEC command is wrapped in a longer statement. This statement provides simple error trapping – if the SQLEXEC command returns a non-positive value, the command that didn’t work is appended to a (pre-existing) variable that can be examined later. The complete code for this single table conversion is contained in CH12C.PRG.

Putting the INSERT in context A couple of points about this very simple example are in order. 1. Remember that because the DBF in this example is not contained in a DBC, there is a limit to how long the field names in the DBF can be. In this example, the VFP and MySQL field names are the same, but that might not always be the case.


2. This first pass assumes the VFP data will fit into the MySQL fields. If the contents of a DBF field won’t fit into the matching MySQL field, the INSERT will fail for that record. It doesn’t matter what the field lengths are, just whether or not the data in those fields fits. In other words, if a VFP field is 50 characters wide and the corresponding field in MySQL is only 35, an INSERT will work IF the ALLTRIMmed contents of the VFP field are no longer than 35 characters. 3. Even though the MySQL table’s iidbiz primary key is defined as ‘auto-increment’, we’re not taking advantage of that feature. Instead, we’re stuffing the primary key value from the VFP DBF. We’re assuming the primary and foreign keys for all of the tables are good, and thus want to keep them in the MySQL version of this database. When a MySQL primary key field is already set up as an auto-increment, you don’t have to include a primary key value in your INSERT. MySQL will create one for you. However, since we’re moving data from an existing database, we don’t want brand new PKs generated, as they certainly wouldn’t match the PKs in the existing data. And that would be bad because the foreign keys in your data would then point to the wrong record. For example, the LOC table has a foreign key, iidbiz, that relates the location record to a parent BIZ record. Multiple locations can all belong to the same parent business. If we let MySQL generate new primary keys for the BIZ table, it is possible, and even probable, that the foreign keys in LOC (which stayed the same during the migration) would now point to the wrong records in the BIZ table. All it takes is one BIZ record INSERT to fail – every key from that point on runs the risk of being incorrect. Fortunately, if you explicitly include the PK as one of the fields in your INSERT, the auto-increment function in MySQL is overridden. So even though it might look funny, the primary key values in the old table are included in the INSERT because you definitely want to move the keys from your old table to your new one. (Note that this doesn’t work the other way around – inserting records into a VFP table can’t overwrite the auto-incremented PK as it is read-only. As a result, you might find the ability to manually populate an auto-increment field a bit unusual.) 4. Finally, you also need to remember to convert VFP dates to four digit years with SET CENTURY ON and VFP times with AM/PM to 24 hour format with SET HOURS TO 24. This is important for the tadded field, for example.

A bit of complexity Not all migrations are as easy as those with one-to-one field mappings. Depending on how well normalized the source (and target) databases are, you may be in for a big chunk of work to transform the data from one structure to another. Obviously, the permutations are many and delving into that topic isn’t the focus of this book. However, they all boil down to needing to massage the data after you pull it out of the source (VFP DBFs) but before you insert it into the target (MySQL). The data in the VFP LOC table has the entire street address in one field, cStreet, while the MySQL LOC table has separate fields for each piece of the address – number, direction, street name, and so on. So this provides a simple, but pointed example of where you would start to add transformation code. use LOC scan m.lcStreet = alltrim(loc.cStreet) * dig the street number out


m.lcNo = substr(cstreet, 1, at(" ", cstreet)-1) * dig the street direction out m.lcDir = iif(at(" S ", cstreet, 1) > 1, ; "S", ; iif(at(" N ", cstreet, 1)>1, ; "N", ; iif(at(" E ", cstreet, 1)>1, ; "E", ; iif(at(" W ", cstreet, 1)>1, ; "W", ; "") ; ) ; ) ; ) * make sure the direction is one of the enum values m.leDir = iif(inlist(m.lcDir, "E", "N", "S", "W"), m.lcDir, "") * dig the street out if at(",", cstreet,1) < 1 * no second line in loc.cStreet, so can just grab the last space m.lcStreet = substr(cstreet, at(" ", cstreet, 2)+1, ; rat(" ", alltrim(cstreet), 1)-at(" ", alltrim(cstreet), 2)-1) else * there's a second line in loc.cStreet m.lcStreet = substr(cstreet, at(" ", cstreet, 2)+1, ; at(",", alltrim(cstreet), 1)-at(" ", alltrim(cstreet), 2)-1) endif * dig the suffix (Rd, Blvd, Ave) out if at(",", cstreet,1) < 1 * no second line in loc.cStreet, so can just grab the last space m.lcSuf = substr(cstreet, rat(" ", alltrim(cstreet), 1)+1) else * there's a second line in loc.cStreet m.lcJustStreet = substr(cstreet, 1, rat(",", cstreet)-1) m.lcSuf = substr(m.lcJustStreet, rat(" ", alltrim(m.lcJustStreet), 1)+1) endif * enum the result m.lcSuf = iif(inlist(m.lcSuf, "Ave", "Blvd", "Cir", "Cyn", "Dr", ; "Lane", "Rd", "St", "Way"), m.lcSuf, "x") * dig the second line out of cStreet, if it exists m.lcsecline = iif(at(",", cStreet, 1) > 0, ; substr(cStreet, at(",", cStreet,1) + 2), "") text to m.lcStr textmerge noshow insert into ins.loc (iidloc, iidbiz, cno, edir, cstreet, csuf, csecline, ccity, cstate, czip, cadded, tadded, cchanged, tchanged) values (<>, <>, '<>', '<>', ; '<>', '<>', '<>', ; '<>', '<>', '<>', '<>', '<>', '<>', '<>') endtext m.lcStrErr = m.lcStrErr + iif(sqlexec(m.liH, m.lcStr) < 1, ; "Command '" + m.lcStr + "' failed. "+chr(13), "") endscan use in LOC

The entire program, with the LOC table transformation code, is found in CH12D.PRG.


Sample migration with multiple data types Even the simple tables that we’ve been migrating contain a variety of VFP data types – character, integer, date, and memo. These can have idiosyncrasies that can drive you nuts. Let’s take a look at a few of them.

Dealing with special characters As you saw from the earlier example, the insert clause for a character field looks like this: '<>',

If biz.cnabiz contains Last National Bank

then the string looks like this when it’s evaluated: 'Last National Bank',

But what if the field contains a delimiting character, like so: Last National Bank o' Nowheresville

Then the string, when evaluated, looks like this: 'Last National Bank o' Nowheresville',

and in the context of the complete SQL INSERT string, the SQL engine will get confused, not knowing which of the quotes terminates the string. The INSERT will fail. The way to get around this is to double the delimiter (known as ‘escaping’ the character), like so: 'Last National Bank o'' Nowheresville',

Doubling the delimiter tells MySQL that the delimiter should be treated as a literal. The trick, then, is to transform each character string with a user-defined function that takes care of this: function Z_ES() * Escape String function lparameters m.tcStringToEscape, m.tcDelimiter if pcount() = 0 return '' else m.tcStringToEscape = alltrim(m.tcStringToEscape) if pcount() = 1 m.tcDelimiter = "'" else m.tcDelimiter = m.tcDelimiter endif endif return strtran(m.tcStringToEscape, m.tcDelimiter, m.tcDelimiter+m.tcDelimiter)


This function takes a string, and, optionally, a delimiter, and returns the string with the delimiter duplicated. If no delimiter is passed, a single quote is assumed. Surrounding each character string with this function will ensure that embedded quotes don’t mess up the INSERT command. The new version of the insert string will look like this: '<>',

As a result, all of the character fields will be surrounded by the z_es() function before being stuffed into the MySQL table.

Dealing with dates Another thing to mention is dealing with empty dates. MySQL will not accept empty dates, but will accept a date of ‘0000-00-00’. That’s not a valid date in VFP, though, so you’ll never find ‘0000-00-00’ in the source of a VFP to MySQL migration. Rather, you need to test the field for emptiness and then pass either the non-empty date as a string, or the ‘0000-00-00’ string. '<>',

If you had more than, oh, one of these guys, you’d probably want to use a UDF to pass the date to and return the appropriate value. Something like the “Test For Empty Date” function: *********************************************************** function Z_TFED() *********************************************************** * Test For Empty Date lparameters m.tdDate if pcount() = 0 m.lcRetVal = '0000-00-00' else * this handles regardless if input is date or date-time m.lcRetVal = iif(empty(m.tdDate), "0000-00-00", ttoc(m.tdDate)) endif return m.lcRetVal

Its use looks like this: '<>',

The full Insert routine looks like this: use BIZ scan text to m.lcStr textmerge noshow insert into ins.biz (iidbiz, cnabiz, cnasec, cadded, tadded, cchanged, tchanged) values (<>, '<>', '<>', '<>', '<>', '<>', '<>') endtext m.lcStrErr = m.lcStrErr + iif(sqlexec(m.liH, m.lcStr) < 1, ;


"Command '" + m.lcStr + "' failed. "+chr(13), "") endscan use in BIZ

The complete code for this program is contained in CH12E.PRG.

Handling BIT fields The one thing this insert statement does not do is handle bit fields. While the basic syntax is fairly simple: insert into database.table (iidpk, bflags) values (1, b'1010')

understanding how to configure the value b’1010’, and then how to retrieve it to display useful information, is more than what would be appropriate for this chapter. As a result, Chapter 13 covers BIT fields in all their glory.

Adding error trapping to the migration process In a perfect world, your program would run without a hitch the very first time, as well as every time after that, regardless of what changes you made to it. Your program would also save the townsfolk, shoot the bad guys, and ride away into the sunset with the dame. Just like mine always do. But in the event that you’re not living in a perfect world, it would be prudent to add error handling to the migration. Our first version already showed some simple error handling in the form of feedback if a command failed. This is important to add to your routines, even your simplest ones, because unlike many other types of programs, when a SQLEXEC() function fails, it does so silently. And twenty minutes later, when you’re scratching your head about why a complex SELECT isn’t working on your database, you realize that there’s no data in the database because your migration failed! Error trapping raises three issues. First, when an error occurs, how do you want to be alerted? Messagebox? WAIT WINDOW? Log to a file? Your choice is partly pragmatic (you don’t want to display a MessageBox every time an INSERT fails on a routine that fails for 337,000 records) and partly personal preference. Second, what do you want to have happen? Should the program be stopped, or should it just ignore that row and do the rest? Finally, once you’re done kicking yourself for “Yet Another Programming Error,” how do you determine what went wrong? How do you go about debugging it?

Alert about the error While you can use a MessageBox or a WAIT WINDOW command to alert you when an error jams up the works of your migration program, I’d advise against it. Even if your program is throwing an error just once in a while, it quickly becomes a nuisance to have to mouse or keyboard past it every time. I’ve already incorporated the timeless ‘debugout’ command that just indicates a SQLEXEC() command failed for quick and dirty alerts. (I keep my Debug Output window open in the lower left corner of my VFP desktop all the time.) In some cases, this is all you’ll need. If you’re doing a set up for a new database (like Ch12B.PRG), or for a small migration of a few hundred or thousand records, sending the results to the Debug Output window, with


its ability to scroll back and forth, as well as to save to a text file (right-click in the Debug Output window and select “Save As...”), is plenty. The decision as to whether (or how long) you stick with DEBUGOUT depends largely on how much output you’re going to spit out. The volume of output is a function of two factors – how much information you produce with each DEBUGOUT command, and how many times DEBUGOUT is called (in other words, how many errors!) For a simple migration program, you may not need output any fancier than what I’ve done here – identifying an error and roughly categorizing it. For larger or more complex migrations, though, you may want to provide more robust handling, such as calling AERROR and include specific information from the columns of the array it populates. (More on incorporating AERROR in your program in a moment.) The number of errors produced depends both on how good your coding is and how good your data is. If you end up with two errors after moving 14,000 records, DEBUGOUT may serve you well. If, on the other hand, you end up with 3700 errors for those 14,000 records, DEBUGOUT may not provide the same amount of useful information. First of all, there’s a limit to how far back you can scroll back in the Debug Output window. And, in a more practical sense, if you have to deal with that many errors, you’re not going to want to visually parse a 60 page text file – if you try to, you will likely miss some problems hiding in the depths of page 42 of that printout. A more useful approach would be to write those 3700 error records to an external error log, preferably in the form of a database table, which you can work through with the tools of your choice. (VFP, cough, cough, VFP.) And you’ll want to take advantage of AERROR’s information when doing so.

Using AERROR() in a program As mentioned in Chapter 6, the AERROR function is used to retrieve ODBC errors. Here’s how to call AERROR upon failure of a call to SQLEXEC, in a program (instead of interactively, as was discussed in Chapter 6.) scan m.liResult = sqlexec(m.liHandle, m.lcStr) if m.liResult > 0 * success! m.lcStrGood = m.lcStrGood + "Insert of record # " + transform(recno()) ; + " was a success!" else * mmmm, bummer, bummer m.lcStrErr = m.lcStrErr + "Insert of biz " + transform(recno()) ; + " FAILED" local aOops[1] m.liHowManyErrors = aerror(aOops) *** *** more code to handle the data stuffed into aOops goes here *** endif endscan

The two new lines follow the m.lcStrErr statement. The first line defines the aOops array (‘aOops’ is just whimsical – you can name the array anything you want.) The second captures the ODBC error information and stuffs it into the aOops array, automatically resizing


it with additional rows as needed. ODBC problems often throw more than one error, and AERROR() returns an integer that represents the number of errors. Since we’re going to stuff every error into its own row in a log table, it’ll be handy to capture how many rows are in the array right up front. The columns in the AERROR() array are described in the VFP Help, but to save you the time looking them up, here they are again:

Table 1. Contents of the AERROR array Column Type 1 2 3 4 5 6 7

N C C C N N C

Description Always = 1526 Error message ODBC error message (frequently similar to #2) ODBC SQL State ODBC data source error number ODBC connection handle Always = .NULL.

Obviously, just creating an array full of error information isn’t going to be helpful by itself. First of all the contents will be overwritten for each call to AERROR, which will happen for each migration record that causes a problem. Furthermore, the array will disappear when the program finishes up. You will probably want to store the contents of the array after each call to AERROR for examination after the program has run its course. I send the errors to a table named ZOOPS (all my system tables begin with ‘Z’ so they sort at the bottom of the listing, out of the way, where they belong.) The structure of ZOOPS looks like this: Structure for table: ZOOPS.DBF Field Field Name Type Width 1 IIDZOOPS Integer (Autoinc) 4 2 INOERR Integer 4 3 CTEXT Character 150 4 CTEXTODBC Character 150 5 CSQLSTATE Character 25 6 INOERRSQL Integer 4 7 IHANDLE Integer 4 8 TADDED Datetime 8 ** Total ** 350

Dec

Index

Collate Nulls No No No No No No No No

Next Step 1 1

The columns in the table match the columns in the array except for the last one; somehow I didn’t think it was that necessary to store a character string of “.NULL.” in every log record. I’m sure I’ll regret it later, but I’ll run off that bridge when I come to it. I also have a single timestamp field, tadded, because it’s handy to know when the record was added. A time changed timestamp isn’t included, because this table typically isn’t modified, and username fields aren’t either, since the only user is the ‘system’. The expanded version of the ELSE segment now looks like this: * mmmm, bummer, bummer m.lcStrErr = m.lcStrErr + "Insert of biz " + transform(recno()) + " FAILED." local aOops[1] m.liHowManyErrors=AERROR(aOops) for m.li = 1 to m.liHowManyErrors


insert into ZOOPS (inoerr, ctext, values ; (aOops[m.li,1], aOops[m.li,5], next

; ctextodbc, csqlstate, inoerrsql, ihandle, tadded) ; aOops[m.li,2], aOops[m.li,3], aOops[m.li,4], ; aOops[m.li,6], datetime())

If you have a large migration and processing time is a concern, you may want to remove the m.lcStrGood/Err statement to speed things up. The complete version of this program, applied to just the BIZ table, is contained in CH12F.PRG. To test the error trapping, get rid of the z_es() function on the ‘<>’ string so records that contain apostrophes generate errors and insert data into ZOOPS.

What should the program do? Now that you’ve taken care of documenting the error, it’s time to think about what your program should do when an error is encountered. You have two decisions to make. The first decision is whether to continue processing, recording which records caused problems, or to immediately terminate upon the first error throwing its ugly head in the air. The types of errors that are commonly thrown in your environment will drive your decision. One set of errors require data cleansing, such as duplicate primary keys coming into the system, or dirty data (fields containing values that the user swore would never happen!). In these cases, you may want to flag the bad records and continue processing. Upon completion, clean up the data and rerun the migration to hit just the bad data. If, on the other hand, the errors showing up usually involve programming mistakes, such as typos in field lengths or incorrect field types, you may want to terminate immediately, fix the problem, and rerun the migration. Catastrophic errors, such as the connection failing after someone spilled an “Enormous Gulp” on the server, may also fall into this category. The second decision is what to do with the processing that has already taken place. If you’re terminating the program immediately, you may want to remove all of the data that you just migrated and rerun from the beginning again. On the other hand, if your migration terminated four records from the end of a two million record conversion, it might behoove you to keep everything and just rerun those last four records. Obviously, the combinations of these two decisions begin to spawn a number of possible scenarios, and it’d be impossible to generate detailed examples for each situation. However, a description of the steps involved may prove useful when developing the migration program your scenario requires. If you’re going to terminate the migration immediately and then rerun from scratch, the only thing you need to remember is to purge the original output table, fix the problem, and then rerun the program. The program CH12B.PRG in this chapter does exactly that by dropping the tables completely. If your tables are huge and you’re only having problems with one table, you would want to modify the routine to drop just selected tables. If you’re going to terminate immediately but run from place where you terminated, you need to plan in advance. You need to be able to start the migration program at a given location in a table, and identify which table if multiple tables are involved. Then you need to be able to save the place in the source table upon an error occurring. Once an error is thrown, fix the problem and rerun the migration program, passing the table and record location as parameters so the program doesn’t reprocess records that have already been handled.


Finally, if you are going to continue processing regardless of errors being thrown, even more preparatory work is required. First, the source data needs to have a flag added that indicates whether the record was successfully migrated or not. Then you need to set up your migration program to process only records that haven’t been flagged as migrated. Once migration starts, each successfully processed source record needs to be flagged as migrated so it doesn’t get processed upon subsequent passes by the migration program. Even if errors occur during migration, the program is allowed to run through to completion, and is then fixed. Finally, the migration program is run again, only touching records that haven’t yet been migrated successfully. The advantage to this scenario is that the process can be run repeatedly, and each time it finishes more quickly, as only the unflagged records are dealt with. And now, the rest of the exercise is left up to the reader.

Debugging the error In the previous section, one critical part of the process was glossed over with three magic words, much like the classic “then a miracle occurs” cartoon: “Fix the problem.” There are several common problems that you’ll run into, and they’re generally easy to figure out from the error messages served up by AERROR(). No database selected

This error happens when you connect to the MySQL server without specifying a database, and then attempt to work with a table without identifying the database that the table belongs to, like so: m.liH=sqlstringconnect( ; + "DRIVER={MySQL ODBC 3.51 Driver};" ; + "SERVER=localhost;UID=root;PWD="+m.tcPW)

and = "insert into biz (iidbiz, cnabiz, cnasec, ) " ;

It can be corrected by connecting to the server and specifying the database, like so (see the third line): m.liH=sqlstringconnect( ; + "DRIVER={MySQL ODBC 3.51 Driver};" ; + "SERVER=localhost;database=ins;UID=root;PWD="+m.tcPW)

or by specifying the database along with the table name: = "insert into ins.biz (iidbiz, cnabiz, cnasec, ) " ;

Although this is ill-advised for more than a guaranteed quick-and-dirty use. Why? If you’re going to do the migration multiple times, you may end up creating multiple databases (ver_1, ver_2, ver_3, etc.), and you’d have to change your code to reflect the new database


name each time. And you know that somewhere in there, you’d forget to change one name and end up writing to the wrong database for one table. A solution partway between specifying the name in the connection string and hard-coding it with every table reference is to make the database name a constant at the top of the routine. Another error you could encounter is: Duplicate entry '23' for key 1

This one, again, makes sense – the source table (in VFP) and the target table (in MySQL) both have a record with the same primary key. This can easily happen if you try to insert records from VFP into MySQL more than once. It’s also possible that your data in VFP is munged; how you fix the problem depends on the cause of the duplicate keys. A third error you might run into is: Data too long for column 'cna' at row 1

This shows up when the field in VFP contains a value that is longer than the length of the field defined in MySQL. For example, suppose the MySQL cCity field is ten characters, and the corresponding VFP field contains the string “Rio de Janeiro.” The attempt to insert a 14 character string into a ten character field will produce this error message. Note that the actual size of the VFP field doesn’t matter – it’s just the size of the string being inserted that counts. As a result, one record’s contents may work while the next won’t. Thus, this error can happen intermittently as the table is scanned. The error A table must have at least 1 column

shouldn’t show up very often, but will happen if you try to create a table without defining any columns (happens to us all...). And finally, if you misspell a table or refer to a table that doesn’t exist (typically when the CREATE TABLE statement failed), you’ll get Unknown table 'bizz'

where ‘bizz’ is the name of the table being referred to.

Not all errors are bad! Before getting into the next section, where we’ll build a more complex SQL INSERT statement with all sorts of interesting data types, let me mention that there are times when an error is not a bad thing – in fact, there could be times when you expect the error. For example, when you’re running a migration from scratch, you might want to create the table all over again, instead of assuming that the table exists. If the table does exist, however, creating the table again will generate an error. So, you could, at the beginning of your migration routine, attempt to create the table and ignore a “table X already exists” error if it occurs. Alternatively, and this is my preference, you could drop the table at the beginning of the migration routine, and then create it again. This has the advantage of also deleting the data, which, if you’re going to be starting from scratch, is something you’d want to do anyway. (The TRUNCATE command will rid the data but keep the table. However, at some point


you’re likely to have structural changes that a DROP TABLE and a subsequent CREATE TABLE will handle properly.) So here is what a slice of code would look like: m.lcStr = "drop table ins.biz" m.liX = sqlexec(m.liH, m.lcStr) debugout iif(m.liX>0, "Success", "Failure:" + transform(m.liX)) if m.liX > 0 * success else * failure local aOops[1] m.liHowManyErrors=AERROR(aOops) for m.li = 1 to m.liHowManyErrors insert into ZOOPS ; (inoerr, ctext, ctextodbc, csqlstate, inoerrsql, ihandle, tadded) ; values ; (aOops[1,1], aOops[1,2], aOops[1,3], aOops[1,4], ; aOops[1,5], aOops[1,6], datetime()) * display an obnoxious alert if the drop table command * generated an error other than the expected 'unknown table' if !("unknown table" $ lower(aOops[1,2])) messagebox("The command" + chr(10) + chr(13) ; + " " + m.lcStr + chr(10) + chr(13) ; + "generated an unexpected error." + chr(10) + chr(13) ; + aOoops[1,2], "Whoa!") endif next endif

If, for some reason, the “drop table” command didn’t generate an “unknown table” error, the user would be alerted with a message box. For example, if you were having “one of those days” and tried the “delete table” command (instead of “drop table”), you’d get an error like that shown in Figure 2.

Figure 2 . Alerting the user about an unexpected error. Of course, the way you want to handle an unexpected error may differ according to your circumstances.

Error handling is just the start Data migration is hard. The development of a data migration application starts in parallel with the development of the new database, and is built in step with the primary application. On many systems, it’s a project unto itself, even with its own staff. And as a result, it’s not just error handling that gets logged, but everything that happens during the process.


As the data migration application is built, one table at a time, a test migration can be run, over and over again, that produces a log that lets you confirm that each step of the process is working correctly. While the development team may get frustrated with having to interface with the migration team, the purpose is to help the developers along by giving them tools and feedback that the data going into the new system is good data.

Reorganizing the migration code You’ve seen that we started to use the same constructs over and over again; that should alert you that something might need more work. There’s an alternate method that will allow you to create a single routine through which multiple SQL commands can be executed. The downside is that you lose the ability to use TEXT/ENDTEXT.

Handling SQL commands that are executed once The basic idea is to put all of the SQL commands into one section, and then all of the standard code – that we have found ourselves beginning to write over and over again – into another section. There are two advantages to this architecture. The first is that the standard code becomes reusable, and as such, is written and debugged once and not messed with again. Always a good thing. The second is that we’ll eventually be able to move the custom code out of the program completely and into a table where it can be maintained without having to modify and recompile the program. Changing the migration simply becomes a matter of updating records in a table. The core concept is to assign all of the SQL strings in the migration to elements in an array called, say, aStr. The first column in the array contains an abbreviation of the command to be used in troubleshooting and the second column contains the entire command, like so: aStr[1,1] = "create database" aStr[1,2] = "create database ins" aStr[2,1] = "create table" aStr[2,2] = "create table ins.biz (" + "iidbiz integer unsigned not null + "cnabiz char(50) not null default + "cnasec char(50) not null default + "primary key(iidbiz)) " ; + "engine = MYISAM"

; auto_increment, " ; '', " ; '', " ;

While you don’t get the benefit of TEXT/ENDTEXT for longer commands, it’s not as big of a deal because these commands are typically going to be one-offs, not subject to a lot of modification. This array could, in the future, become records in a table. Next you spin through the array, executing each row with a standard routine, using SQLEXEC. The execution routine now looks like this: for m.li = 1 to alen(aStr,1) m.liX = sqlexec(m.liH, aStr[m.li,2]) if m.liX < 1 debugout "The command " + aStr[m.li,1] + " failed." return endif next


It’s little trouble to make a change to a SQL command or add a new SQL command to the routine; just modify the array as needed. The rest of the program stays the same. It’s even easier if you decide to put your SQL commands into a table – just add another record to the table before processing. Similarly, when you want to enhance the execution routine (the block of code where SQLEXEC() is actually called), you just need to do it in one place. For example, if you want to replace the dullard DEBUGOUT command with the witty and sophisticated AERROR trap, you only need to do it once.

Handling SQL commands that are executed multiple times This mechanism, with an array and a loop that spins through the array to execute each command, works great if you execute each command just once. W hat if you need to do a command multiple times – say, when you’re spinning through a table and hitting record after record? You may think you could simply write a SQL INSERT command like so: aStr[n,1] = "INSERT into BIZ" aStr[n,2] ; = "insert into ins.biz (" + "iidbiz, cnabiz, cnasec " ; + ") values (" ; + alltrim(str(biz.iidbiz)) + "," ; + "'" + z_es(hero.cnabiz) + "'," ; + "'" + z_es(hero.cnasec) + "'" ; + ")"

Then you’d wrap the SQLEXEC command in a loop for each row in the table and execute it in a subroutine like so: m.lcNumRecs = transform(reccount()) scan wait window nowait "Processing record # " + transform(recno()) ; + " of " + m.lcNumRecs if sqlexec(m.liH, aStr[m.li,2]) > 0 * success! else * (AERROR handling omitted for sake of clarity) endif endscan

There is a slight hitch with this process, though. The SQL INSERT command, when passed to the execution subroutine, has already had the table values incorporated into the statement, so the string being passed to SQLEXEC() looks like this: m.lcStr = "insert into ins.biz (iidbiz) values (1)"

for every record! Useful for record #1, not so useful for all the others. Instead, we want to pass something like this: m.lcStr = "insert into ins.biz (iidbiz) values (" ; + alltrim(str(biz.iidbiz)) + ")"


so the value of hero.iidhero is subbed in at runtime. There are two changes we have to perform in order to make this possible. First, we change the assignment command where we assemble the SQL INSERT, creating a literal string with the variable code as part of the string. Then we use VFP’s EVALUATE() function in the execution subroutine to cause the variable to be evaluated at runtime (in SQLEXEC()) instead of when the INSERT command is being assembled. Here’s the new INSERT statement: aStr[n,1] = "INSERT into BIZ" aStr[n,2] ; = ["insert into ins.biz (] ; + [iidbiz, cnabiz, cnasec ] ; + [) values ("] ; + [ + alltrim(str(biz.iidbiz)) + "," ] ; + [ + "'" + z_es(biz.cnabiz) + "'," ] ; + [ + "'" + z_es(biz.cnasec) + "')" ]

Enclosing the statements in brackets creates a string (brackets are a string delimiter just like single and double quotes) that won’t be evaluated until explicitly evaluated with the “evaluate” function. Now the SQLEXEC() function will look like this: if sqlexec(m.liH, evaluate(aStr[m.li,2])) > 0

While this is an interesting technique you might want to keep stored in the back of your mind, it doesn’t solve the problem of not being able to use TEXT/ENDTEXT to make the formatting easier. This is a trade-off that you’ll have to reconcile. The reason you can’t use TEXT/ENDTEXT inside a construction like this is because the string inside the TEXT/ENDTEXT delimiters is a literal: text to m.lcStr insert into ins.biz (iidbiz, cnabiz) values (<>, '<>') endtext

This type of code, unfortunately, won’t work: aStr[1,1] = "insert into biz" aStr[1,2] = "insert into ins.biz (iidbiz, cnabiz) ; values (<>, '<>')" aStr[2,1] = "insert into person" aStr[2,2] = "insert into ins.person (iidperson, cnaperson) ; values (<>, '<>')" for m.li = 1 to alen(aStr,1) text to m.lcString textmerge noshow &aStr[m.li,2] endtext next

The string “&aStr[m.li,2]” will be sent to m.lcString as is. Fortunately, there is a corresponding function to TEXT/ENDTEXT called textmerge() that will do the trick for us: for m.li = 1 to alen(aStr,1)


m.lcString = textmerge(aStr[m.li,2]) next

Passing strings to a SQLEXEC() wrapper can cause other problems as well.

Passing long strings A common problem when passing a string to the wrapper is ending up with a string that’s too long. The error you get from MySQL is confusing, too – it tells you that you have a syntax error in your SQL command. This is because VFP has a limit of 255 characters for a string literal, and it’s easy to build a CREATE TABLE or INSERT statement that’s longer than that. VFP silently truncates the command at 255 characters and passes the resulting string to MySQL, which then sees just part of a command and tells you that there’s a syntax error. The following command, when passed to SQLEXEC(), generates a “Command contains unrecognized phrase/keyword” error: m.lcStr = ["create table ins.biz (] ; + [iidbiz integer unsigned not null auto_increment, ] ; + [cnabiz char(50) not null default '', ] ; + [cnasec char(50) not null default '' ] ; + [primary key(iidbiz)) ] ; + [engine = MYISAM"] ? sqlexec(m.liH, evaluate(m.lcStr))

However, when you break up the string into smaller pieces, like so (note the plus sign and quotes in the fifth line, and the quotes at the end of the fourth line): m.lcStr = ["create table ins.biz (] ; + [iidbiz integer unsigned not null auto_increment, ] ; + [cnabiz char(50) not null default '', ] ; + [cnasec char(50) not null default '', " ] ; + [+ "primary key(iidbiz)) ] ; + [engine = MYISAM"] ? sqlexec(m.liH, evaluate(m.lcStr))

the code runs fine. Finally, you may be tempted to try to include the sqldisconnect() in the array. In the heat of the battle, it’s easy to assume that SQLDISCONNECT is just the last in a long line of commands being passed to SQLEXEC(). However, it isn’t, any more than SQLSTRINGCONNECT() is! So it stays out of the array.

Moving more than data So far the preceding discussion has focused strictly on the data itself. What about all of the other stuff that goes along with a database, such as indexes as well as properties that belong to the table – record and field validation, triggers, relational integrity, mapping fields to classes, input masks, captions, and so on? Some of these don’t have a place in a MySQL database while others, with the addition of stored procedures to MySQL 5.0, can possibly be moved over. However, doing so will be a manual process, because VFP’s SQL implementation isn’t exactly the same as MySQL’s. You will also most likely need some custom modification of your VFP routines before they’ll work in MySQL. Of course, that’s the same story whenever you’re moving stored procedures from


one database to another, and one way database vendors attempt to achieve lock-in – it’s not the dollar cost that prevents conversions, but the horror of trying to rewrite large numbers of stored procedures. Of course, if you’re going to use remote views, they need to be stored in a DBC, which means you’ve still got access to stored procedures as well as DBC events.

Performance issues As mentioned in earlier chapters, the choice between using MyISAM and InnoDB tables comes down to the need for performance versus the need for transactions. If you absolutely need transactions, then you don’t have a choice. However, if you’re waffling between the two, wondering how much faster a MyISAM table is than an InnoDB table, here are a few interesting data points. I ran a few simple tests on identical MyISAM and InnoDB tables. Each table was in its own database, located on the local hard disk. I ran a routine to insert 100,000 100-byte records into the MyISAM table, and then again in the InnoDB table. I also used two different syntaxes – the first creating a handle to MySQL, but not explicitly connecting to a database, and then including the database name as part of the INSERT command. Then I ran the whole process again, but this time explicitly identifying the database when creating the handle to MySQL, so I didn’t have to explicitly reference the database name in the INSERT command. Not naming the database in the connection string used these commands: m.liH=sqlstringconnect("DRIVER={MySQL ODBC 3.51 Driver};SERVER=localhost;UID=root;PWD=secret") insert into superheroes.hero (iidhero, cnaf, cnal, csecretidentity) ; values (" + alltrim(str( m.li )) + ",'" + 'al' + "','" + 'alvin' ; + "','" + 'alvin anderson' + "')"

Naming the database during the connection used these commands: m.liH=sqlstringconnect("DRIVER={MySQL ODBC 3.51 Driver};SERVER=localhost;UID=root;PWD=secret;database=superheroes") insert into hero (iidhero, cnaf, cnal, csecretidentity) ; values (" + alltrim(str( m.li )) + ",'" + 'al' + "','" ; + 'alvin' + "','" + 'alvin anderson' + "')"

All times in the following table are in minutes:seconds.

Table 2: Relative MySQL Performance during INSERTS MyISAM

InnoDB

Naming Database in Connection String

2:03

59:10

Not Naming Database in Connection String

2:55

62:03

Quite a difference! The MyISAM INSERTS were hugely faster than the InnoDB INSERTS. It was also interesting to see that the explicit connection to a database saved nearly a third of the time for


MyISAM (52 seconds), while the amount of time saved for InnoDB was greater (2:53) but percentage-wise, was just a fraction of the total time, about 5%. Another interesting piece of information is the relative sizes of the files created. 100,000 records in MyISAM file structures (two files, a table, and an index) ended up being 4,449,280 bytes (3,200,000 in the table and 1,249,280 bytes in the index), while the single InnoDB ibdata1 file was nearly twice as large: 8,388,608 bytes. Obviously, this isn’t an exhaustive benchmark (I didn’t do queries, for example), but it shows you that there are scenarios where there is a significant difference between the two engines as well as the connection techniques. If you will be running a migration process repeatedly, it makes sense to do some testing of various mechanisms to take advantage of speed differences when appropriate. Running a migration that takes all day harkens back to the days of submitting a job (in boxes of punch cards) to the computer desk and coming back the next day, hoping it ran without errors.

Conclusion/Summary Getting your VFP data into your MySQL database is an important part of moving to MySQL as a backend for your VFP applications. In this chapter, we looked at the second of two of the most common mechanisms for doing so, and discussed some performance issues as well. Now that you have the tools to create production quality databases, it’s time to start building the VFP front-end for those databases. Updates and corrections to this chapter can be found on Hentzenwerke’s Web site, www.hentzenwerke.com. Click “Catalog” and navigate to the page for this book.

Chapter 13: Advanced Data Issues 249

Chapter 13 Advanced Data Issues Once you’re comfortable with creating MySQL databases and populating them from VFP tables, you’ll spend some time doing so and may possibly run up against some issues more sophisticated than simply filling character and numeric fields as you did in the last chapters. In this chapter, I’ll cover tips and tricks with BIT fields, Blobs, Nulls, and ENUMs.

Many databases consist of just a few datatypes – characters, numbers, and dates. Other types of applications have a wider range of needs, resulting in data types that don’t have direct analogs in VFP.

BIT field operations A BIT field is, in opposition with its name, a collection of one or more bits, the number being user-defined. As mentioned in Chapter 11, you can store multiple logical fields in a single BIT field, using each bit as a true/false flag. The first thing to discuss is when to use BIT fields and when not.

Strategic use of BIT fields to replace VFP logicals As you will see shortly, it’s not very hard to use a single BIT field to store values for multiple logical fields. However, just because you can doesn’t mean you should. It’s still not trivial, but more importantly, using a single, generically named bit field, like ‘bflags’, can easily lead to confusion because it’s not self-documenting. Who can remember if the third bit was for this logical and the fourth was for that logical, or if it was the other way around? In my view, it’s considerably easier to spelunk through a database six months or six years later with individual BIT(1) fields named ‘lIsNew’ and ‘lOnHold’, rather than looking for the documentation that, darn it, was just around here the other day. If you use individual BIT(1) fields, you don’t have to go through the rigmarole of using VFP’s BITTEST to determine if there’s a 1 or a 0 in the field. Suppose you have a BIT(1) field named lOneFlag in the variousbits.tabletwo table. The SELECT command SELECT * FROM variousbits.tabletwo WHERE NOT lOneFlag

will pull all records where lOneFlag is 0. You can use this syntax directly in MySQL (like in the Query Browser) as well as passing it to MySQL via VFP’s sqlexec() function. Ted Roche reminds us that this works well when you are absolutely certain the field is going to contain only two values – Yes and No, True and False, On and Off, and so on. If you’re facing the possibility of the field’s domain values expanding to Yes, No, Pending, and Unknown, then a different field type would be in order. One such choice would be the ENUM field type, which is covered later in this chapter.

250


A bit arithmetic and functions primer In order to make best use of our time with BIT fields, it’d be a good idea to refresh our understanding of bit arithmetic, and then learn about VFP’s and MySQL’s bitwise functions. Let’s look at a ‘bit’ of bit arithmetic. A bit field contains a string of 1’s and 0’s, like so: 10101100

Each digit represents a power of 2, starting from the right. Thus, ‘1101’ would be 1*2^3 + 1*2^2 + 0*2^1 + 1*2^0

or 8+4+0+1 = 13. You can perform operations on a bit field using special ‘bitwise’ functions. Both MySQL and VFP have these functions, although the two programs have somewhat different sets. Since VFP’s are slightly richer, in terms of what we’re going to want to do, I’ll use them. The logical OR function (thought of as ‘either one or the other or both’) takes two bits as operators, and returns a 0 if both bits are 0s, and returns a 1 in any of the other three cases (the first is 1 but the second is 0, the first is 0 but the second is 1, or both are 1, thus the derivation of ‘either one or the other or both’). To wit: 1 0 1 0

OR OR OR OR

1 1 0 0

= = = =

1 1 1 0

BITOR operates on two values, each of which is a series of bits, and performs a logical OR on each bit in the series. For example, BITOR(1000, 1001) returns 1001. The leftmost bits in both series are a 1, and a logical OR on two 1’s results in a 1. The next bits in both series are a 0, and a logical OR on two 0’s results in a 0. The rightmost bits in both series are a 0 and a 1, and a logical OR on a 0 and a 1 results in a 1. Here are a couple more examples: BITOR(1111,0000) BITOR(1010,0101) BITOR(0000,0001) BITOR(0001,1000)

returns returns returns returns

1111 1111 0001 1001

The BITAND function performs a logical AND on two values. In this case, the corresponding bits in both values have to be 1 in order for the function to return on 1. Examples: BITAND(0000,1111) BITAND(1100,1100) BITAND(1010,0101) BITAND(1000,0001) BITAND(0110,0010)

returns returns returns returns returns

0000 1100 0000 0000 0010

You can try these out in VFP via the command window. However, you can’t type in ‘1001’ as a binary value; instead, you need to convert it to a boring old decimal number. So, 0101 would be 5 (0+1+0+1), and 1111 (8+4+2+1) would be 15, but you knew that, right? Try this:


nnnn = 0 nyny = 5 yyyy = 15 ? bitor(nyny,yyyy) 15 ? bitor(nnnn,yyyy) 15 ? bitand(nyny,yyyy) 5 ? bitand(nnnn,yyyy) 0

Why the funny variable names? Because the “n” characters represent 0’s and the “y” characters represent 1’s, and thus it’s visually easy to figure out what binary values you’re working with. Of course, you still have to noodle through the return value, but that’s just one conversion you have to do in your head, not three. Now that you’re comfortable with the basics, let’s look at two more functions that we’ll actually use shortly. BITTEST will determine whether a given bit in a string is a 1, which is a function that MySQL doesn’t have, and is going to be darn handy. It takes two values; the first value is the series of bits to be tested, and the second is the position of the bit in the series to be tested. Note that the offset starts from the right, and begins with 0. Thus, to test bit ‘b’ in the string ‘abcd’, you’d use bittest('abcd', 2)

and if b was a 1, the return value would be .T.

As with the BITOR and BITAND functions, you need to pass the series of bits as a decimal number too. Again, in the VFP command window: yyny=13 ? bittest(yyny,0) .T. ? bittest(yyny,1) .F. ? bittest(yyny,2) .T. ? bittest(yyny,3) .T. ? bittest(13,1) .F. ? bittest(13,2) .T.

&& test the rightmost 'y' && test the 'n' && test the 'y' to the left of the 'n'' && test the leftmost 'y'

Finally, the BITSET and BITCLEAR functions allow you to poke a 1 (bitset) or a 0 (bitclear) into a specific bit position in a series of bits. For example, let’s set the variable ‘uuuu’ (“unknown”) to 0, since we’re going to fiddle with it and the bits will keep changing. uuuu=0

252


This means the bit representation is 0000. (Actually, it’s many, many 0’s, but four is enough for our purposes.) Now use the BITSET function to set the third bit (counting from the right) to a 1. We pass a ‘2’ as the second parameter because the offset is zero-based. The return value is 4. ? bitset(uuuu,2) 4

The value ‘4’ is, in binary, 0100, which means we successfully set the third bit from the right. Now let’s set the second bit: ? bitset(uuuu,1) 2

You may be thinking that the return value should be 6, not 2, since we now have set both the second and third bits. But that’s not true – our first bitset function didn’t change the value of ‘uuuu’, it just used it as input. How’s this? uuuu=bitset(uuuu,2) ? uuuu 4 uuuu=bitset(uuuu,1) ? uuuu 6

And BITCLEAR works the same way. Clearing the second bit from the right will turn uuuu back to 0100, or 4: uuuu=bitclear(uuuu,1) ? uuuu 4

There are a few more bit-oriented functions, but these are all that we’ll need in order to handle BIT fields with VFP and MySQL.

Define a BIT field As discussed briefly in Chapter 10, you define a BIT field like so: bit(4) bit(13) bit(64)

where the parameter is the number of bits in the field. 1 is the smallest allowable value and 64 is the maximum. ‘bit(1)’ means you can store either ‘0’ or ‘1’, while ‘bit(4)’ means you can store bit strings like ‘1000’ and ‘1111’. A TINYINT field is the same as bit(1). In a SQL CREATE TABLE statement, bit fields would look like this: create database variousbits create table variousbits.tableone (bhasvoted bit(1)) create table variousbits.tabletwo (cname char(10), bflags bit(8))


Putting data into a BIT field The following SQL statements will insert a bit string into a BIT field: insert into variousbits.tabletwo (cname, bflags) values ("al", b'00000001') insert into variousbits.tabletwo (cname, bflags) values ("bob", b'10001000') insert into variousbits.tabletwo (cname, bflags) values ("carla", b'11110000')

And the following will update BIT fields: update variousbits.tabletwo set bflags = b'00001111' where cname='carla'

Rather straightforward, once you learn the syntax of the binary values. The complete code for this creation is in CH13A.PRG and a new file, Z_SQL.PRG, contains the library functions.

New functions for modularizing code There are a couple of new features in this program that I’ll take a brief side journey to discuss.

Using a common SQLEXEC function: z_sqlexec() Back in Chapter 12, I mentioned the technique of assigning commands that would only be executed once to an array, and then calling the elements of that array in a common function that held the actual SQLEXEC command. Here is an example of how you could do it. The array, aStr, holds all of the commands to drop and create the VARIOUSBITS database and its tables, and then insert test values. Once the array has been populated, a simple FOR/NEXT loop is used to iterate through the list of commands in the array. A call to a new library function, z_sqlexec(), is the gist of the body of the FOR/NEXT loop: =z_sqlexec(m.liH, aStr[m.li,2], aStr[m.li,1])

The z_sqlexec() function, in this example, is fairly simple function z_sqlexec() * wrapper for SQLEXEC() * sample call: * =z_sqlexec(m.liH, aStr[m.li,2], aStr[m.li,1]) lparameters m.liHandle, m.lcStr, m.lcError * used for single SQL statements if sqlexec(m.liH, m.lcStr) > 0 * success else local aOops[1] m.liHowManyErrors=AERROR(aOops) =z_sqlerror(@aOops, m.lcStr) endif return .t. endfunc

There’s really more overhead than fancy programming, but shortly we’ll see that it provides the structure of a more flexible SQLEXEC handler. The other thing it does right now is allow the inclusion of a call to a common error trapping routine, explained next.

254


Using a common error trapping routine: z_sqlerror() In Chapter 12, I incorporated the AERROR error handling in the main routine. While that was fine for a first example, it’s not practical to repeat that code for each call to SQLEXEC. Better to put it in a library function and call it as needed. You can see the call to z_sqlerror() in the previous listing for z_sqlexec(). The code for z_sqlerror() looks like this: function z_sqlerror() lparameters aOops, lcStr m.liHowManyErrors = alen(aOops,1) for m.lii = 1 to m.liHowManyErrors insert into ZOOPS ; (cStr, inoerr, ctext, ctextodbc, csqlstate, inoerrsql, ihandle, tadded) ; values ; (m.lcStr, aOops[m.lii,1], aOops[m.lii,2], aOops[m.lii,3], aOops[m.lii,4], ; aOops[m.lii,5], aOops[m.lii,6], datetime()) next return .t. endfunc

Not tricky or sophisticated, but it does centralize the code and allows changes to be made in a single place.

Adding a procedure file The z_sqlerror() function, as well as functions introduced in Chapter 12, z_es() and z_tfed(), are all now contained in a separate procedure file, Z_SQL.PRG. The SQLEXEC wrapper, z_sqlexec(), is not in the procedure file because we’re going to enhance it shortly. It stays in the main program for the time being.

Get data out of a BIT field Putting data into a BIT field is easy; getting it out is a little trickier. Well, getting it out is easy, getting it out in an understandable format, well, that costs extra! Let’s fire up the Query Browser and try out a few things. If, after you executed the three INSERT statements in the previous section, you do a SQL SELECT in the Query Browser, you’ll get results that look like Figure 1.


Figure 1. Displaying the contents of a BIT field in the Query Browser. Disappointing, to say the least. How are you supposed to make out the difference in the ‘bflags’ field values in the various rows? Here’s the trick – you can use special syntax wrapped around your BIT field name to display the contents of the BIT field in various incarnations, like so: select cname, bflags, bflags+0, bin(bflags+0), oct(bflags+0), hex(bflags+0) from variousbits.tabletwo

The result will display as shown in Figure 2.

256


Figure 2 . Displaying the contents of a BIT field in decimal, binary, octal, and hex. As you can see, the various wrappers for the BIT field name display the contents in decimal, binary, octal, and hexadecimal. Note that the content isn’t changed – just the way it’s displayed to our human eyes. (Also note that ‘00000001’ in binary displays simply as ‘1’.) The display is reproduced in the following table as well. +------+------+------+----------+----------+----------+ |cname |bflags| dec | bin | octal | hex | +------+------+------+----------+----------+----------+ |al || | 1 | 1 | 1 | 1 | |bob || | 136 | 10001000 | 210 | 88 | |carla |a | 240 | 11110000 | 360 | F0 | +------+------+------+----------+----------+----------+

Now that we can get data in and out of a BIT field, how do we manipulate just a single bit in the field? The BIT field contains multiple logical flags, each are the MySQL version of a VFP logical field. What if we want to find out what just one of the flags is? Or perhaps we want to change just one of the many logical flags that the BIT field represents. How?

To extract just one position In order to find out what just one of the fields in a BIT field is, use the VFP BITTEST function. The following code segment shows the essential commands required. m.liFlagNumber = 1 declare aStr[1,4]


aStr[1,1] = "get bflags" aStr[1,2] = ["select cname, bflags, bflags+0 as b_dec, ; bin(bflags+0) as b_bin, oct(bflags+0) as b_oct, hex(bflags+0) as b_hex ; from variousbits.tabletwo where cname = 'carla'"] aStr[1,3] = "csrFlags" aStr[1,4] = "s" m.li = 1 =z_sqlexec(m.liH, aStr[m.li,2], aStr[m.li,1], aStr[m.li,3], aStr[m.li,4]) m.llIsTrue = bittest(val(csrFlags.b_dec),m.liFlagNumber-1) messagebox("The value of flag #" + alltrim(str( m.liFlagNumber)) ; + " is " + iif(m.llIsTrue, "True", "False"), "BITTEST Results")

In order to determine the value of a different bit, change “m.liFlagNumber” from 1 to a different position in the series. How you want to make use of the value of the flag once it’s been dug out of the table – select a checkbox, disable a control, execute a branch in a program – is up to your needs in your application.

To input just one position MySQL doesn’t have a native BITSET-like function like VFP, so there are two ways to set the flag for just one bit. The first is to use a combination of the available bit functions in MySQL to ‘roll your own’ BITSET function, but since we already have one in VFP, why go to all that trouble? The second, and much easier, way is to grab the existing value from MySQL, manipulate it inside VFP, and then stuff the new value back into MySQL. Here’s an example. Let’s suppose we want to flip the second flag in the ‘bflags’ field for carla. First, we need to get the original bflags value. Then we manipulate that value with a Visual FoxPro BITSET function. Finally, we stuff the new value back into MySQL. And we do all of this via SQLEXEC() functions. Here are the essential commands: m.lcStr="select cname, bflags, bflags+0 as b_dec FROM variousbits.tabletwo " ; + "where cname = 'carla'" m.liNewValue = bitset(val(csrFlags.b_dec),1) m.lcStr = "update variousbits.tabletwo set bflags = " ; +alltrim(str( m.liNewValue )) ; + " where cname = 'carla'"

The first command assigns the decimal value of the bflags field to ‘b_dec’. Then the VFP function BITSET is used to set the second flag. Remember that the offset starts with 0, and BITSET automatically changes the value to ‘1’ regardless of what it used to be. And the last command stuffs the new decimal value into MySQL. The complete code for this routine is contained in CH13B.PRG.

New functions for creating cursors Again, there are a couple of new pieces in this program – CH13B.PRG – that I’ll take a moment to discuss.

SQLEXEC()’s third parameter The first new feature is that SQLEXEC() takes a third parameter that we haven’t talked about yet. The third parameter is useful only if you’re doing a SQL SELECT (which, admittedly,

258


might be rather often). The parameter is the name of the cursor that the results of the SELECT are placed into. If you don’t include the third parameter, VFP automatically creates a cursor named ‘SQLRESULT’. You should get into the habit of specifying your own cursor, because as soon as you issue a second SELECT, the contents of the original SQLRESULT cursor are gone, and more likely than not, you’re going to want them around. The only downside to creating your own cursor names is that it can be easy, if you’re not meticulous about cleaning up after yourself, to end up with a bunch of used-only-once cursors lying around.

Breaking up aStr If you examine CH13B.PRG, you’ll see that the FOR loop that iterates through the aStr array to send each command to z_sqlexec() to be executed is gone. Instead, while the array is still used to hold the various commands, z_sqlexec() is manually called for each row in the array. As you can see, intermediate processing (called the BITSET function) is required between some of the commands. Why, you may ask, do we bother with the aStr construct at all, then? The reason is that by still calling z_sqlexec(), we keep all of the goodness of the error trapping and handling that z_sqlexec() provides us, and that allows us to focus on one place for our custom code – right up top.

When to use BLOB fields to store files Open a discussion about BLOB, TEXT, Memo, and General fields and invariably a debate arises – over whether you store files (PDFs, images, whatever) in the database itself, or if you store each individual file on disk, and use a character field to point to the location of the file. The technical details were discussed in Chapter 10. But that doesn’t impart wisdom. There are several issues to consider, including the infrastructure used to store the database and the files, how permanent the files are, whether the database engine can do incremental backups, and the size of the files.

Infrastructure Using a character field in the table to point to an individual file on disk provides a lot of flexibility, and with a bit of cleverness in the naming scheme, an application can be given a path and then pointed to a variety of files. A dynamic naming scheme also provides for the ability of the application to scan for related content, such as thumbnails of named image files. On the flip side, if an app has to be deployed over a WAN or have a Web interface, both situations where a simple drive and folder naming scheme won’t work, it can be trickier, to the extent of being impossible to accurately point to discrete files.

Permanent and incremental backups Ron Gafron, an early reader of this manuscript, having worked with imaging applications for many years, says that one issue is whether the files are archival – not changing once they’re inserted – or active – where the file itself may change multiple times over the life of the database. Next, consider whether or not your database can do incremental backups. If you have a database that is primarily archive-related, and your database can’t do incremental backups,


you’ll end up backing up the entire database, which means you’ll be backing up the same, unchanged data over and over. As file-containing databases can get large, the backup becomes a lengthy process, which adds risk to the process. By just using pointers to files, the database size is greatly reduced, which speeds up the backup. You can simply back up the files separately. If you have a database that can do incremental backups, the problem is minimized, but if your database gets large, storing a large number of files, database performance can still be negatively affected.

File size Finally, it depends on how big the files you’re storing in the database are. Storing a hundred thousand 50K PDFs is a lot different than storing a hundred thousand 10 MB photographic quality images. The bottom line is that there’s no single, one-size-fits-all answer. You need to balance the trade-offs and requirements of your application and environment.

Dealing with NULLs You can live a long, long time in Visual FoxPro without ever having to think once about NULL values. The original DBF file structure had no place for them; and as many VFP developers trace their roots from pre-NULL days, they developed habits (sometimes bad) for dealing with situations where a NULL would have been the appropriate mechanism. So let’s brush up on NULLs for a moment, because with MySQL, they’re much more important.

What is a NULL? NULL means “I don’t know.” Suppose you had a table where each row recorded the temperature of a city on a given day. One day the Internet goes down partway through the transmission process of data from each city to the location of the database. So while you have temperatures for every city east of the Rockies, Portland, San Francisco, Fresno, Salt Lake City, Phoenix, and Palm Springs were out of luck and didn’t get a chance to report. What values do you put in the temperature field for those cities? You don’t put ‘0’, because ‘0’ is a valid temperature (and pretty unlikely to be the right temperature for Palm Springs). Instead, you use NULL, because you don’t know what the temperature is. By the way, if you’re thinking that you could simply leave the field empty, tsk, tsk, tsk. In some fields, an empty value is a legitimate value as well. An example would be middle name – there’s a difference between not having a middle name – empty – and not knowing what one’s middle name is – NULL. The tricky part of NULLs is that when you do calculations that incorporate NULLs, the answer is always unknown, because if the calculation involves a value that you don’t know, you can’t know the result either. For example, suppose you’re trying to answer “What is the average temperature across the nation?” The answer has to be “I don’t know,” because you don’t know what the temperature was for all of those cities, so you don’t know the average. This doesn’t mean you won’t get an answer back if you try – the answer will just be wrong. Here’s a quick example (if you don’t want to type this in, the complete code for this routine is contained in CH13C.PRG.) CREATE CURSOR temps (cCIty c(25), nTemp n(5,2) NULL) INSERT INTO temps VALUES ("Los Angeles", 72.3)

260

INSERT INSERT INSERT INSERT SELECT


INTO temps INTO temps INTO temps INTO temps avg(nTemp)

VALUES ("Las Vegas", 95.2) VALUES ("Bismarck", -73.8) VALUES ("San Juan", 68.0) VALUES ("Milwaukee", NULL) FROM temps && returns 40.43

The value returned is 40.43, which isn’t necessarily correct, because there is no data for one of the cities. (75.2+95.2-73.8+68.0+NULL)/5 = I don't know!

Yes, if you asked the question differently, such as “What is the average recorded temperature across the nation?” you could answer, but that question means you could legitimately throw out all of the records with NULL values. OK, enough pontificating. Whenever you get confused about NULLs, just replace 'NULL' with the magic words, “I don’t know.”

Setting up a NULL When creating a definition for a field, you have two choices, either mark a field as NOT NULL, or don’t mark it so. In other words, if you don’t explicitly mark a field as NOT NULL, you can later put NULL values in the field. Consider it the lazy programmer’s way of defining databases. So we have sort of ‘the lady or the tiger” situation here. What happens in each case? If you mark a field as NOT NULL, you must provide a default value (except for autoincrementing integers). Doing so means you will have known values in every field, which then means you can do searches and arithmetic on the contents of the field. If you don’t mark a field as NOT NULL, you’ll be allowing nulls into your data. Each time you insert a record into a table, and don’t explicitly provide a value for that field, a NULL will be put in the field automatically. If you’re not expecting this behavior, things can get ugly sooner than you’d think. Ugh. The reason many programmers allow NULLs in their table definitions is that they don’t want to do the upfront thinking of what a legitimate default value would be. Instead, they defer that work until later, when they’re going to have to deal with a database full of “I don’t knows.” (But maybe by that time, they’ll be working on a different project!) Let’s look at some of the possibilities to help you reconcile those here-to-fore unthoughtof issues. Character fields can usually be left blank, thus, the default value would be ‘‘ (two single quotes). It’s not often that you’ll need to know when a character field’s blank truly represents the legitimate absence of a data value, versus when it means you don’t know. One example is a name and address table that has unknown middle names. (Perhaps the Social Security Administration would need to know, in the event that an application was smudged, making the middle illegible. But even in that type of situation, it’s likely that there would be a standard placeholder value, such as “unreadable!”.) Date/datetime fields can often be stuffed with ‘0000-00-00’ as a default (although not with the blanket abandon for character fields). It’s pretty evident that ‘0000-00-00’ is not a legitimate value, and serves to say “I don’t know” without causing all of the problems that NULLs in the table can induce.


Numeric fields are more problematic, because one often does arithmetic on these types of fields. (Imagine that!) It’s a problem because 0 is a legitimate value, and thus can’t represent the possible absence of data like a blank in a character field can. Is the number of times someone has visited a certain exhibit in that Museum Visitor Tracking database truly ‘0’? Or do we simply not know? So, to sum up, what should you do? It depends on your situation. If you have a reason to know when you have unknown values in a field (such as temperature tracking), mark the field as NOT NULL and provide an empty default value. Having to figure out what default value to use in those rare cases that a blank isn’t appropriate is much better than allowing NULLs to be entered into fields, and then having to do deal with the “I don’t know” question all over the place (which you will!). In most cases, having empty default values in a field will be acceptable. If you have marked your field to allow Nulls you can use the ISNULL() MySQL function in your SELECT statements to filter the results.

ENUM fields Enum fields (enum is short for “enumerated”) are conceptually easy to grok, and a wonderful tool to keep garbage out of your tables. Defining a field as enum allows you to control what values go into the field, at the database level. No code needs to be written, and no bytes will be harmed during the filming. However, the syntax can initially be tricky for VFP developers, since there isn’t an enum field type in Fox. To create an enum field via the SQL CREATE TABLE command: create table mytable (edir enum(' ', 'E', 'N', 'S', 'W') not null) create table another table (edir enum(' ', 'Credit', 'Debit') not null)

One suggestion often made on the MySQL forums is to provide a blank value in the list of enumerated values. This provides the ability to display a blank in the field. Otherwise, you would have to choose one of the other values. In this example of directions on the compass for street addresses, there are many addresses that don’t have a directional. Including a blank as one of the enumerated values allows you to accommodate those addresses. Another suggestion regularly seen is to put the values in the list in alphabetical order, as it will enable you to sort on the field. Internally, MySQL doesn’t store the value (say, “N” or “Credit”) in each record. Instead, MySQL stores an index value. This index value matches the position of the value in the field definition, so in the lines above, ' ' has an index value of 1, then ‘E’ has an index value of 2, and so on. This is important because these index values are used when the field is indexed. Thus, if you had placed the ‘W’ before the ‘E’, the index value of the ‘W’ would be smaller than the index value of the ‘E’, and thus records with ‘W’ values would show up in front of ‘E’ records; a result you’re probably not interested in achieving.

Conclusion/Summary One advantage of MySQL over Visual FoxPro is a more robust set of data types. In this chapter, we looked more in-depth at some of the new data types in MySQL, and how to work with them. Updates and corrections to this chapter can be found on Hentzenwerke’s Web site, www.hentzenwerke.com. Click “Catalog” and navigate to the page for this book.

262


Chapter 14: Constructing SQL to Avoid SQL Injection 263

Chapter 14 Constructing SQL to Avoid SQL Injection Most people have heard the term “SQL Injection” but are often unconcerned about the subject, feeling it’s a security problem – and since they’re just database developers – it couldn’t possibly apply to them. Even if you’re building applications today where the threat of SQL Injection isn’t applicable, it behooves you to learn more about it and to develop best practices. So, if you do enter that arena in the future, you’re not suddenly vulnerable due to long-developed habits that are suddenly inappropriate in a new environment. In this chapter, I describe what SQL Injection is, when you might run into it, and how to program defensively to avoid it.

SQL Injection. The term sounds fairly sophisticated, probably something we don’t have to worry about quite yet, right? Here’s a typical scenario that will send chills down your spine. The user navigates to a Web application (public, private, it doesn’t really matter – employees are as big a security risk as outside hackers). For the password, they entered a string of garbage like this: notverysecret" or 0=0

and boom! They’re in the system, able to browse accounts, edit data, and generally do anything that a legitimate user could do. They’ve been authenticated and are in the application. If you’re putting an application together with a backend SQL database (be it MySQL or a competitor), this could happen to you if you’re not paying attention.

How SQL Injection works SQL Injection is a security vulnerability that exposes an application to attacks through the database engine. It occurs when SQL commands in an application are created or altered by the insertion (or “injection”) of unexpected characters by external agents. For example, when input from a user is not validated properly, it can allow ‘bad’ data in the form of unexpected SQL commands or clauses that allow improper access to the database or execute against the database. SQL injection can achieve various attacker goals in vulnerable systems, including gaining privileged access, altering data, and denial of service. Web applications are especially vulnerable because of the inability to enforce control over user agents (browsers, etc.) and the character-based nature of Web transactions. There are several forms of SQL Injection. The exact code needed to demonstrate a problem varies with the application’s language as well as the back-end database used, so to illustrate here I’ll use pseudo VFP and MySQL code in the following examples.


Improperly trapped delimiters The bogus password problem discussed at the beginning of this chapter is an example of a poorly trapped delimiter. Suppose the code used to validate users included a statement like so: m.lcStr = "select * from USERS where un = '" + m.lcUserName

+ "'"

(ignoring the password for the time being, to keep the example simple). The result set would be a single record containing user data for an authenticated login. From this, rights for access to the application could be garnered, and all is good. Supposedly. However, if a malevolent user entered a specially crafted username, like this: m.lcUserName = [' or '1-' = '-1 ]

(note the trailing space), the SELECT statement substitutes the value like so: m.lcStr = "select * from USERS where un = '" + "' or '-1' = '-1" + "'"

which becomes this: m.lcStr = "select * from USERS where un = '' or '1-' = '-1'"

The result set was supposed to be just a single user with a validated username (and, in real use, a password), but, of course, ended up being all users (since ‘ 1’ is always equal to ‘1’, right?) The code then just grabbed the first record (since there would only be one, no need to check how many were actually in the result set, right?), and determined the rights of the user. So a bogus user has gotten in and secured access to the application.

A working SQL Injection example How about an example to see how this really works? Smashing idea! The source code for this chapter includes two programs that demonstrate a simple SQL Injection exercise. The program, CH14A.PRG, sets up a sample database, SQLINJ, with a table named ‘user’ that contains one field (character type, length 20). Then the program adds four records to ‘user’. The second program, CH14B.PRG, takes a parameter that is used to find a specific record in the ‘user’ table. The first two parameters passed to CH14B are the MySQL username and password, as has been done in previous examples. The third parm acts as a username that you’d enter, for example, into a Web page to gain access to the system. The code that is executed looks like this: * user entered value (might be good or bad) m.lcStr = "select * from sqlinj.user where cna = '" + m.tcAppLogin + "'" =z_sqlexec(m.liH, m.lcStr, "", "csrUser_one_rec") debugout "User-entered string >" + transform(m.lcStr) + "<"

The third parameter, m.tcAppLogin, is compared to the values in the ‘user’ table in order to authenticate the visitor. If at least one match is found, a cursor is created, csrUser_one_rec, that contains all matches. There should be only one match. Finally, the text string containing the SQL command is displayed in the Debug Output window.


If you don’t pass a third parameter, one is appointed for you. Two SQL commands are executed one after another, each creating a separate cursor to display the results. The first command uses a good value (one that matches just one record in the ‘user’ table). The result should be a cursor with just one record. The second command uses a bad value that causes a SQL Injection. The result is the entire table being returned in the cursor, as described earlier. * one good value m.tcAppLogin = [al] m.lcStr = "select * from sqlinj.user where cna = '" +m.tcAppLogin+"'" =z_sqlexec(m.liH, m.lcStr, "", "csrOne_rec") debugout "Good string >" + transform(m.lcStr) + "<" * now show results with a bad value m.tcAppLogin = [' or ' ' = ' ] m.lcStr = "select * from sqlinj.user where cna = '" +m.tcAppLogin+"'" =z_sqlexec(m.liH, m.lcStr, "", "csrAll_recs") debugout "Malformed string >" + transform(m.lcStr) + "<"

In both cases, the text string containing the good (or bad) SQL command is displayed in the Debug Output. You can try calling CH14B with your own variations on the third parameter, like so: do CH14B with 'bob', 'secret', 'some nefarious, malformed value'

The complete code for this example is contained in CH14A.PRG and CH14B.PRG.

Direct injection: failure to validate data Suppose your application has a textbox named ‘SomeVar’ on a form. After the user enters data into the textbox, they click a command button to execute code in your application. This code looks something like this: m.lcVar = GET('SomeVar') m.lcCommand = "SELECT a,b,c from MyTable where SomeField = " + m.lcVar =sqlexec(m.lcCommand)

Typically, your user would enter “Maine” into the SomeVar textbox, and your application would then dig out all of the bed and breakfasts in Maine. But what if the user entered the string Maine";delete from MyTable where 1=1;

When SQLEXEC() executed the m.lcCommand string, the SELECT executes and digs out all Maine records, as expected. Then the ‘delete’ command is executed, and whump. Goodbye, data. Note that this doesn’t work with current versions (5.1.x) of MySQL. For example, in the previous example, if you set the third parameter like so: m.tcAppLogin = [';update sqlinj.user set cna='george' where cna='donna]

both commands (the SELECT and the UPDATE) would fail.


Quoted injection: failure to trap data type Subtly different from the last form of attack, the failure to trap for the proper data type can also result in unexpected results and/or inappropriate data being sent to the SQL engine. Suppose your MySQL command expected a numeric data type, like so: "select * from CUSTOMER where CUST_NO = " + iCustNo + ";"

If the malevolent user entered a string, such as 1000;drop database CUSTOMER

it’d be time to find some new customers. Or some new programmers.

Database engine defects Hard as it may be to believe, occasionally a database engine has a bug in it. In these cases, it can be possible to craft an input that takes advantage of the defect. This is akin to a buffer overflow, where an attacker takes advantage of poor coding in the program itself. The gist of the attack is for a malformed or specially crafted input string to be submitted to the SQL engine that then has unintended (and usually adverse) effects. A few years ago, a MySQL exploit targeted Windows servers running MySQL. It took advantage of a feature of the engine that allowed a file to be written to disk. Since Windows operating systems lack executable permission attributes, the file, after being written, could be run and then take over the MySQL installation. Of course, to avoid “fingerprinting” by hackers, your application should not reveal what database engine is being used. Nevertheless, many hackers with automated tools will attempt these exploits, on one system after another, without even knowing if systems have a specific vulnerability. (Take a look at Linux Web server’s Apache logs and see how many attempts to run c:\winnt\cmd.com are listed.) While harder to find and exploit, they can also be more dangerous, because the problem isn’t as readily obvious as the previous forms of SQL Injection described.

Solutions to SQL Injection There are several ways to combat SQL Injection. While these aren’t strictly necessary for client-server applications, they do provide best practices and the habits you develop will serve you well in the future. So this is what SQL Injection is – injecting bad or incorrect data into a SQL statement that then gets processed in a manner that you don’t want or hadn’t anticipated. The solution, then, should be obvious. Actually, there are several solutions.

Validate the data The first line of defense is to validate the data. This means to take the input and confirm that it is the proper data type; that the user isn’t typing gibberish characters, and so on. As a result, you don’t want to just take the user’s input and let your application execute with it as part of a command. Instead, you validate the input. Here’s a simple example. m.lcVar = allt(lower(GET('SomeVar'))) if m.lcVar == "new hampshire" ;


or m.lcVar == "maine" ; or m.lcVar == "vermont" * carry on! else * 404 the user endif

In many cases, however, it isn’t practical or possible to validate every possible value that a user might enter. You can provide more robust trapping via code that looks for a variety of unacceptable conditions. First, consider trapping the length of the value, like so: m.liMaxAllowableLength = 14 && longest state name is 14 characters if len(m.lcVar) > m.liMaxAllowableLength * bad input endif

Next, trap for allowable characters. Can you restrict input just to numbers? If someone is entering a date, there’s no need to accept alphabetic characters. Can you restrict just to numeric and alphabetic characters? If you’re entering a postal code, there’s no need to accept parens or ampersands, but you might need to deal with hyphens. So if you need to accept special characters, explicitly accept just those you know you will need, and deny the rest. In some cases, you can ignore (i.e. silently strip out) unwanted characters (such as additional quotes in a text search form), while in other cases, you may want to take more aggressive action, such as raising a validation message or even alerting system administrators. Regular expressions (a string that describes or matches one or more strings, according to predefined rules, that can search and manipulate bodies of text based on certain those rules and patterns) can greatly facilitate validation in Web applications. By creating patterns for what user input is expected (called a ‘white list’) or unexpected (called a ‘black list’), you have more power and flexibility than what is available by only checking individual characters. Detecting unwanted input helps protect your application, but you must also decide on the proper strategies to employ when such input is encountered. Is bad input an indication of potential mischief, or simply an innocent mistake by a fumble-fingered typist? This discussion is starting to get a little far-field from the issue at hand, which is protecting our MySQL application from bad input. Fortunately, there’s a better, more farreaching technique.

Parameterized queries A parameterized query takes the form of a WHERE clause with one or more variables preceded by a question mark, like so: m.lcVar = allt(lower(GET('SomeValue'))) m.lcSQL = "select * from CUSTOMER where CUST_NO = ?m.lcVar"

Parameterized queries don’t inject the values into the SQL command. Instead, the values are processed via parameter expressions. In other words, the parameter is passed to the backend database server as a piece of data, not as a potential command or clause to be interpreted.


A parameterized query in real life So how does this look in practice? You might be tempted to modify the SELECT command like so: m.lcStr = "select * from sqlinj.user where cna = ?'" +m.tcAppLogin+"'"

and then run the program like so: do ch14c with 'bob', 'secret', 'donna'

The resulting command would look like this select * from sqlinj.user where cna = ?'donna'

which would surprisingly return the correct record set. Unfortunately, it’s still open to SQL Injection. If you pass a bad value (such as the OR clause with empty quotes), you’ll get the same results – all records. Your next attempt might look like this: m.lcStr = "select * from sqlinj.user where cna = '?" +m.tcAppLogin+"'"

and the calling command still looks this: do ch14c with 'bob', 'secret', 'donna'

which produces a command like so: select * from sqlinj.user where cna = '?donna'

but not surprisingly, this will return an empty set, since there are no records in the table whose ‘cna’ field values begin with a question mark. The trick is to build the question mark into the SQL string, passing the entire command including the variable name to the back end. At the same time, the value of the variable has to be available in memory. A simplified version of CH14B, calling SQLEXEC() directly, looks like this: m.lcStr = "select * from sqlinj.user where cna = ?tcAppLogin" =sqlexec(m.liH, m.lcStr, "csrUser_one_rec") debugout "User-entered string >" + transform(m.lcStr) + "<"

The value of m.lcStr that’s being sent to the server is select * from sqlinj.user where cna = ?tcAppLogin

The reason this works is because the value, tcAppLogin, is available in the current routine. Moving this to the z_sqlexec() function requires a bit of modification so the value of m.tcAppLogin is local in z_sqlexec(). If you simply call z_sqlexec() with this command, the server will request the value of the variable, like so:


Figure 1. The back-end server will request a value for the parameter if it’s not available locally. So we have to make sure the value of the variable is available in the z_sqlexec() function. This can be done by passing the value of m.tcAppLogin to z_sqlexec() as a parameter: =z_sqlexec(m.liH, m.lcStr, "parm", "csrUser_one_rec", m.tcAppLogin)

and then in z_sqlexec(), read that variable. Note the actual variable name, m.tcAppLogin, has to be available in z_sqlexec() before sending it to the back-end server via SQLEXEC(). function z_sqlexec() lparameters m.liHandle, m.lcStr, m.lcError, m.lcNaCursor, m.lcVar m.tcAppLogin = m.lcVar * used for single SQL statements if sqlexec(m.liH, m.lcStr, m.lcNaCursor) > 0 * success else local aOops[1] m.liHowManyErrors=AERROR(aOops) =z_sqlerror(@aOops, m.lcStr) endif

(In production version code, you’d probably want to use generically named variables.) Now running the program like so: do ch14c with 'bob', 'secret', 'carl'

results in a single record, as we wanted. Running do ch14c with 'bob', 'secret', "' or ' ' = ' "

results in an empty cursor, again, which is what we wanted. This code is all contained in the file, CH14C.PRG, in the source code downloads for this chapter.

Additional defenses While this book is about MySQL, and this chapter specifically about SQL Injection, it’s worth mentioning a couple more ideas from Randy Pearson as far as validating input into SQL commands, particularly when it comes to Web apps.


•

•

•

•

•

•

First, remember that your data validation routine needs to deal with unexpected characters like carriage returns. While it’s difficult to type a carriage return into a textbox, a carriage return can be dumped into one via a clipboard copy. Next, just because input into a SELECT initially came from a hard-coded control on a Web page, such as an option button or a combo box, it doesn’t mean the value being passed in is a legitimate value. It’s all too easy to read the underlying HTML, replace the value associated with a control’s value, and post the new, modified HTML with an unintended value. Unlike LAN apps where you really do know the underlying value, you can’t be sure on the Web. White lists (where you explicitly allow only specific values) are more secure than black lists (where you explicitly disallow specific values and let everything else in). In other words, it’s better to list what is allowed and deny everything else, than to try to identify every possible item that should be denied. You’re bound to forget or not know about some values. This isn’t always feasible, but the concept is worth keeping in mind. Different back-end databases have different vulnerabilities. For example, one noticeable difference between using VFP as a ‘back-end’ database versus MySQL or SQL Server is the use of the semi-colon (;) as a line continuation character versus a command terminator. Each time you use macro substitution, get up, walk around, and then ask yourself, “Do I really want to do this?” Finally, use variables (or stored procedure parameters) instead of concatenating SQL commands as strings that include user input whenever possible. If variables inside a SQL statement are never ‘quoted’, the command can’t be injected.

Conclusion/Summary SQL Injection is one of the most common methods of attacking databases on the W eb. In this chapter, we explored what SQL Injection is and how it works with respect to VFP and MySQL, and then explained a number of techniques to circumvent it. It’s a best practice to learn more about it and to practice good coding techniques so you’re not vulnerable due to long-developed habits. Updates and corrections to this chapter can be found on Hentzenwerke’s Web site, www.hentzenwerke.com. Click “Catalog” and navigate to the page for this book.

Chapter 15: Religious Wars 271

Chapter 15 Religious Wars: Remote Views, CursorAdapters, and SQL PassThrough With the introduction of CursorAdapters in VFP 8, there are now three different ways to access remote data in Visual FoxPro. How do you choose which one to use? In this chapter, I'll discuss the pros and cons of each mechanism, and explain why I use SQL PassThrough in this book.

So far I’ve been manipulating MySQL data from VFP using SQL PassThrough. As mentioned in earlier chapters, there are two other methods to connect VFP and MySQL: Remote Views and CursorAdapters. Each has its place and each has its fan base. The arguments for and against each can get heated at times (hence, the chapter title, ‘Religious Wars’), but as with many things, the decision comes down to balancing the technical merits of one approach against another with one’s personal preferences.

Remote Views Remote views are where many people get their start with client-server applications. A view is nothing more than a SQL command stored in a DBC, and there are two kinds. Local views connect to VFP DBFs while remote views use ODBC to connect to a back-end database. Both use the same Visual FoxPro functions, such as tableupdate(), that are used to update local tables, which reduces the learning curve for experienced VFP developers who are new to views. Technically, you can create a remote view to link to local data (VFP DBFs), but that’s a rather uncommon scenario. The main reason you’d do so is if you needed an application to work both with local and remote data sources – by having your application work solely with remote views, you can theoretically swap one set of remote views with another. In the real world, it isn’t quite that easy, but it beats bracketing all data access code and using two completely different access methods. Additionally, there’s a performance hit involved if you connect to local DBFs via a remote view’s connection, so you really “gotta wanna” be able to switch.

How to use Ironically, the steps required to produce a remote view are more involved than to create a CursorAdapter or use SPT. From 10,000 feet, the steps include creating a database container, adding a connection, and creating the view. Let’s take a closer look. In this example, we’ll be connecting to the BIZ table in the INS database running on our MySQL server.

Create a DBC Open Visual FoxPro, and issue the command


modify database CH15

An empty database container will display, as shown in Figure 1.

Figure 1. An empty database container. The undocked toolbar shown in Figure 1 includes a button for creating a new remote view, as shown in Figure 2.

Figure 2 . The Database Designer toolbar.

Create a connection Up to this point, we’ve been creating a connection manually through code. With a remote view, you need to create a data source if you haven’t already. The easiest way is via the Data Sources (ODBC) applet in Windows’ Control Panel. You can also create it programmatically by inserting registry keys or importing a .REG (Registry) file that contains the connection information. Then you create a connection that is added to the database container. For this example, we’ll use the data source created via the steps in Chapter 6. The result is a System DSN, as shown in Figure 3.


Figure 3 . An existing Data Source. Now, onto the connection. Click the Connections button in the Database Toolbar (shown in Figure 2) to open the Connections dialog as shown in Figure 4.


Figure 4 . The Connections in the CH15 database container. Click the New button to create a connection for the DBC. The dialog shown in Figure 5 will display.

Figure 5 . A sample connection, using the mysqltest data source.


Click OK, give the connection a name, and the Connections dialog will appear again with the new connection displayed, as shown in Figure 6.

Figure 6 . The Connections in the CH15 database. Close the Connections dialog. Now it’s time to create the remote view.

Create a remote view Click the New Remote View button in the Database Toolbar (see Figure 2.) You are given a choice to use the View Wizard or to create a view manually using the View Designer (shown in Figure 15 later in this chapter.) The following steps walk you through the View Wizard. The first page in the wizard is shown in Figure 7.

Figure 7 . Clicking the Connections option button displays existing connections in the database container.


You can choose from all ODBC data sources or from connections in the database. Be aware that an ODBC User Data Source name may not be available for all users on all systems, while System Data Source names are available to all users on the system, but still must be set up either manually or programmatically on all systems your application is running on. Clicking the Connections option button will display existing connections in the database container. Once you select your data source or connection, select Next to choose tables and fields, as shown in Figure 8.

Figure 8 . The available tables in the database attached to the selected connection are displayed. As you click on each table in the Tables listbox, the fields in that table are displayed. You can use the mover buttons between the Available and Selected Fields listboxes for each table. A typical result is shown in Figure 9.


Figure 9 . Selecting a table displays the fields available for selection. Once you choose your fields, you can control the order they’ll show up in the view with the grey buttons to the left of each field in the Selected Fields listbox. While technically unimportant for a view, it can be handy to have them in a preferred order, say, if you simply want to browse the fields. Once you’re done selecting tables, fields, and the display order, click Next. If you selected fields from more than one table, you’ll advance to Step 3, which allows you to choose how to relate tables. Otherwise, you’ll advance to Step 4, which allows you to determine how the records will be sorted. If you have more than one table, Step 3 allows you to select the key fields from both tables for the join condition, as shown in Figure 10.

Figure 10 . The next step in the Wizard is to choose how the tables relate if more than one was chosen for the view.


Step 3a allows you to choose what type of join to create, as shown in Figure 11.

Figure 11. If you create a join, you can select what type of join to make. Step 4, shown in Figure 12, displays the fields available for sorting.

Figure 12 . The fourth step in the wizard is to choose the sort order of the records in the view. Step 5, not shown here, allows you to select a subset of the view via a filter. Whether you do or not, the final step, Step 6, allows you to choose what to do after completion of the view, as shown in Figure 13.


Figure 13 . The finish step allows you to choose what to do after the view is created. When you save the view, you are prompted for a name (dialog not shown), and then the view is created and a view container is displayed in the database container window, as shown in Figure 14.

Figure 14 . The database designer displays the new view.

Modify the view Once you create it, you can open the remote view in the View Designer. Right-click the view in the database container and select “Modify” from the context menu. The View Designer will open, as shown in Figure 15.


Figure 15 . The View Designer allows you to modify an existing view. Each step in the wizard is represented by a tab in the lower pane of the View Designer. In addition, the Update Criteria tab allows you to create a view that will send changes back to the server, as shown in Figure 16.

Figure 16 . The Update Criteria tab allows you to set a view to be updatable. Setting up a view to be updatable is a bit tricky. When you initially open a remote view in the View Designer, the columns to the left of the listbox (the columns with the key and the pencil) are blank, and the “Send SQL updates” checkbox in the lower left is disabled.


In order to set a view to be updatable, you must first select a primary key upon which records in the view will be matched with the records on the back-end database. Figure 16 shows an additional field in the view, biz.iidbiz, that provides a primary key for updating. This primary key can be made up of one or more fields. Figure 16 shows the field biz.iidbiz as the key field. Next, you need to choose which fields will update the back end by selecting them in the column under the pencil. Figure 16 shows the cnabiz field will be written back to the back end. Once you select at least one field, the “Send SQL updates” checkbox is enabled. It must be checked for the updates to actually be sent back to the back end. You can generate the code that represents the view via the “SQL” button on the View Designer toolbar. Doing so for the view we just created displays something like the following: SELECT Biz.iidbiz, Biz.cnabiz, Loc.iidloc, Loc.cno, Loc.edir, Loc.cstreet,; Loc.csuf, Loc.ccity; FROM ; {oj biz Biz ; LEFT OUTER JOIN loc Loc ; ON Biz.iidbiz = Loc.iidbiz}; ORDER BY Biz.cnabiz DBSetProp(ThisView,"View","SendUpdates",.T.) DBSetProp(ThisView,"View","BatchUpdateCount",1) DBSetProp(ThisView,"View","CompareMemo",.T.) DBSetProp(ThisView,"View","FetchAsNeeded",.F.) DBSetProp(ThisView,"View","FetchMemo",.T.) DBSetProp(ThisView,"View","FetchSize",100) DBSetProp(ThisView,"View","MaxRecords",-1) DBSetProp(ThisView,"View","Prepared",.F.) DBSetProp(ThisView,"View","ShareConnection",.F.) DBSetProp(ThisView,"View","AllowSimultaneousFetch",.F.) DBSetProp(ThisView,"View","UpdateType",1) DBSetProp(ThisView,"View","UseMemoSize",255) DBSetProp(ThisView,"View","Tables","biz,loc") DBSetProp(ThisView,"View","WhereType",3) DBSetProp(ThisView+".iidbiz","Field","DataType","I") DBSetProp(ThisView+".iidbiz","Field","UpdateName","biz.iidbiz") DBSetProp(ThisView+".iidbiz","Field","KeyField",.T.) DBSetProp(ThisView+".iidbiz","Field","Updatable",.T.) DBSetProp(ThisView+".cnabiz","Field","DataType","C(50)") DBSetProp(ThisView+".cnabiz","Field","UpdateName","biz.cnabiz") DBSetProp(ThisView+".cnabiz","Field","KeyField",.F.) DBSetProp(ThisView+".cnabiz","Field","Updatable",.T.) DBSetProp(ThisView+".iidloc","Field","DataType","I") DBSetProp(ThisView+".iidloc","Field","UpdateName","loc.iidloc") DBSetProp(ThisView+".iidloc","Field","KeyField",.T.) DBSetProp(ThisView+".iidloc","Field","Updatable",.T.) DBSetProp(ThisView+".cno","Field","DataType","C(6)") DBSetProp(ThisView+".cno","Field","UpdateName","loc.cno") DBSetProp(ThisView+".cno","Field","KeyField",.F.) DBSetProp(ThisView+".cno","Field","Updatable",.T.) DBSetProp(ThisView+".edir","Field","DataType","C(1)") DBSetProp(ThisView+".edir","Field","UpdateName","loc.edir") DBSetProp(ThisView+".edir","Field","KeyField",.F.) DBSetProp(ThisView+".edir","Field","Updatable",.T.)


DBSetProp(ThisView+".cstreet","Field","DataType","C(35)") DBSetProp(ThisView+".cstreet","Field","UpdateName","loc.cstreet") DBSetProp(ThisView+".cstreet","Field","KeyField",.F.) DBSetProp(ThisView+".cstreet","Field","Updatable",.T.) DBSetProp(ThisView+".csuf","Field","DataType","C(10)") DBSetProp(ThisView+".csuf","Field","UpdateName","loc.csuf") DBSetProp(ThisView+".csuf","Field","KeyField",.F.) DBSetProp(ThisView+".csuf","Field","Updatable",.T.) DBSetProp(ThisView+".ccity","Field","DataType","C(35)") DBSetProp(ThisView+".ccity","Field","UpdateName","loc.ccity") DBSetProp(ThisView+".ccity","Field","KeyField",.F.) DBSetProp(ThisView+".ccity","Field","Updatable",.T.)

A nice update to Visual FoxPro 9 is that updates to code in this SQL window are saved back to the view – so you have ‘two-way’ editing, a feature that VFP developers for a long time looked longingly at in Borland’s tools.

Pros Remote views are easy to create and use; indeed, the View Wizard allows you to create a remote view without a word of programming. Remote views hide the complexity of connecting to a back end, setting up the commands to dig data out of the server, and return updates. If you’re comfortable with tableupdate(), you can use a remote view. The View Designer has been significantly improved, and is able to handle more and more types of complex views. Additionally, there are third party tools that provide support for views, in some cases improving upon the native functionality of the View Designer, or even improving upon its abilities.

Cons Maintainability of views, despite the improvements in the View Designer, is still a big chore. Complex views still need to be maintained outside of the View Designer. You can only send SELECT statements. A remote view, remember, is simply a SQL SELECT. You can flag it as updatable so changes to the data in the view are sent back to the server. Want to add a record? Add a record to the view and then commit the changes. You don’t use SQL INSERT or UPDATE or DELETE, which can be very handy tools. Worse, you can’t create or alter tables, or any of the other nifty back-end database functions. And even worse than that, you cannot handle or control back-end transactions. You can’t update parent/child tables, or other multi-table scenarios, as a single unit. Views have to be defined ahead of time, and can’t be modified at run-time. Yes, you can send parameters in the WHERE clause, but this ability is very narrow. Suppose you have a query that retrieves all the locations in a certain zip code. You can build a parameterized view so the user can select a value for the zip and that value gets sent to the view: where cZip = ?lczip

However, what happens when you want to include a second clause to the WHERE, say, only those that are flagged as headquarters? Your first choice is to create a whole new view,


with everything the same as the first one, except with a slightly different WHERE clause. Or you can change the WHERE clause, adding a second parameter, like so: where cZip = ?lczip and lIsHQ = ?lIsHQ

While this isn’t complex, as soon as you add one clause, you’ll find you’ll soon need to add a second, and a third. This can (and usually will) get complicated quickly. Views require a DBC stored on the network with the application (not on the server) to hold the view definitions. Connection info must be stored in the DBC, and is part of the view. Views can only perform synchronously (although you can set a property to do background fetching). If you change the schema of your database, you will have to manually change your views. Very ugly when you have 300 views in the DBC and a change to a field affects 70 of them, and this scenario is easy to run into.

CursorAdapters “CursorAdapter” is a shorthand reference to the object created from the VFP CursorAdapter class. The class provides a consistent interface for working with remote data sources, regardless of the underlying connection mechanism – ODBC, OLEDB, ADO, or XML. You create an object based on the CursorAdapter class, and set some properties, such as the DataSourceType (e.g. ODBC), the DataSource (a connection string), and the SelectCmd (“select * from CUSTOMER”). Then you execute the CursorFill method of the object to create a cursor of the subset of data you want from the back end. The benefit of a CursorAdapter is that you can change some properties of the object, such as the data source type or the data source, but end up with the same cursor. Since your application just works with the cursor, it doesn’t have to know anything about how that data came about.

How to use The following simple snippet of code shows you how to create a CursorAdapter for the INS database created in Chapter 12. Specifically, it creates a cursor based on the BIZ and LOC tables in INS. set multilocks on loCursor = createobject("CursorAdapter") with loCursor .Alias = "All our Customer Locations" .DataSourceType = "ODBC" .DataSource ; = sqlstringconnect("DRIVER={MySQL ODBC 3.51Driver};" ; + "SERVER=localhost;DATABASE=INS;UID="+m.tcUN+";PWD="+m.tcPW) .SelectCmd ; = "select cnabiz, cno, edir, cstreet, csuf, ccity, cstate, czip " ; + "from BIZ join LOC on loc.iidbiz = biz.iidbiz" if .CursorFill() debugout "Success in filling " + .Alias + "!" browse fields cnabiz, cno, edir, cstreet, csuf, ccity, cstate, czip else m.liHowMany = aerror(aOoops)


m.lcStrError = '' for m.li = 1 to m.liHowMany m.lcStrError = m.lcStrError + aOoops[m.li,2] + chr(13) next debugout m.lcStrError endif endwith

This is just the interesting part; the entire program is found in CH15A.PRG.

Pros The flexibility of being able to switch from one data source type to another can be useful in certain scenarios. First, if you’re building a vertical market application, you may need to support a variety of back-end databases. Using CursorAdapters to provide a single point of contact with the database makes it easier to do so. Another situation you may find yourself in is when the customer has multiple back-end data stores, either because of disparate acquisitions or because of the old “things are the way they are because they got that way” evolution of multiple systems in a changing environment. Being able to build your application to connect to all of those back ends and present them as if they came from a single source is handy. That said, being able to support multiple back ends is different than the classic case that the marketing droids often pitch – “Hey, what if you boss shows up one Monday morning and demands that you change your back end?” Personally, I’d argue that it’s more of a parlor trick than a highly sought-after feature, compared to, say, active promotion of VFP by Microsoft. How often do you build an application and then find, all of a sudden, that you have to change the back end from ODBC to ADO? That’s a pretty drastic change, and if it’s going to happen, it’s pretty likely that the database schema is going to change as well, which means the UI is going to change, as are the reports... and suddenly the need to be able to change the data source type by setting a single property isn’t going to be a killer chore. And the flexibility of being able to change from one data source type to another isn’t germane to us as VFP/MySQL developers, because MySQL only works with ODBC. Still, this flexibility is still clearly in the “Pros” column for CursorAdapters. The CursorAdapter class has hooks throughout the VFP UI, making it easy to use in the Class Designer as well as with the CursorAdapter Builder. You can set the SelectCmd property to a hard-coded string or dynamically supply it, which provides more flexibility than remote views.

Cons CursorAdapters are limited to SELECT tasks, just like remote views. You can’t perform data definition tasks like creating and altering tables, nor any administrative or security work on the back end. Maintenance can be a real pain, as each class is similar to a remote view, and thus has to be maintained individually. You’ll have to track which CursorAdapter class is related to which parts of the database schema, so when the schema changes, you’ll know which CursorAdapter to change, much like having to maintain remote views by hand.


SQL PassThrough As you’ve seen in earlier chapters, SQL PassThrough (SPT) allows you to explicitly create SQL statements on the client and send them to the back-end server. You can send more than just SELECT commands; the entire range of data manipulation (SELECT, INSERT, UPDATE, DELETE, etc.) as well as data definition (CREATE DATABASE, CREATE TABLE, ALTER TABLE, etc.) commands available on the server are also available to you in your application via SPT.

How to use The concept behind SPT is simple. You create a connection, which returns a numeric value called a “handle.” You then refer to the connection by the handle, and pass strings containing SQL commands, along with the handle, to SQLEXEC() (or other commands), which sends the SQL command to the server to be executed. When you’re done, you clean up after yourself by closing the connection. There’s no need to repeat the code for a sample SPT session as there are plenty of them in the last few chapters.

Pros You have complete, sophisticated, and flexible control over the server – you can send any command that the server understands, including administrative and security functions. SPT commands – queries, table definition changes, anything – can be put together dynamically. SPT provides full support for both asynchronous and synchronous connections. You can use any existing connection on the fly. There’s no need to be stuck with the connection defined with the view, or to manually override the connection as with remote views.

Cons There is no native VFP interface for working with SPT; they’re all hand-coded. There is no UI at all, no wizards, no builders. This is daunting for the first time user. To be sure, there are third-party tools but, sadly, many developers (or their management) have an aversion to using anything that doesn’t come ‘in the box’. Since SPT is all code, it’s easy to end up with an application that has a bit of SPT here, a bit more there, some more over there in the corner, and suddenly when you have to make a change to the underlying schema, you have a nightmare on your hands. It’s possible (and advisable) to put all your SPT code in one place (can you say ‘class library’?), but this is not a trivial exercise. SPT is for people with attention spans. SPT cursors do not automatically update their source, and have no persistent presence. Every time you query a cursor, it is destroyed and recreated from scratch. Cursors used with SPT are created at runtime, and thus, unlike remote views, aren’t available for drag and drop operations during development.

Decision time Years ago (back before there was a ‘2’ in the year), I built a very large client-server system using remote views. It was connected to three separate SQL Servers (one for an off-the-shelf CRM package, a second for an off-the-shelf, but highly customized, accounting package, and a


third for the custom work I did.) This system was used by nearly every person in the company, including many on the factory floor. While the code required was fairly straightforward (the business rules and UI took, by far, the most amount of time), I still spent an inordinate amount of time (I felt) just maintaining the views. As the system was rolled out, the customer kept adding features, and so a large, but manageable number of remote views ended up tripling in count. Robert Green, a former VFP product manager, wrote the first true VFP/Client-Server book, back in the mid-90s. In it, he said, “...remote views for data entry and SQL PassThrough for read-only queries. This is the easiest way of writing a client/server application... As you gain more experience with client/server applications, you should explore the ‘harder’ way of building client/server applications by relying more on SPT and less on remote views.” Andy Kramek concurs. He discusses the problems with ad-hoc queries, saying “essentially it is a query where the filter condition may vary at run time depending on user actions. It is this sort of query that remote views are most emphatically not good at.” Some argue that SPT is difficult or hard-to-use; perhaps for novices it is. However, SPT is extremely powerful – you have the entire server at your beck and call, the only limitation being your mind. As a result, I prefer SPT over the other techniques. I value the power and flexibility that SPT gives me above the limitations of lower ease of use and lack of a UI. If you’re just going to slap together a few quick apps, remote views or CursorAdapters would be fine. But if you’re into this for the long haul, and expect to build a serious application with requirements of robust functionality on the server and long term maintenance, SPT is the way to go.

Conclusion/Summary One of the hallmarks of Visual FoxPro is that there are always a number of different ways to accomplish any task. Obviously, accessing remote data is no exception. If you’re new to client-server apps, you may feel more comfortable with remote views or CursorAdapters, while if you’ve been around the block a few times, SPT might fit you better. Updates and corrections to this chapter can be found on Hentzenwerke’s Web site, www.hentzenwerke.com. Click “Catalog” and navigate to the page for this book.

Chapter 16: A Client-Server State of Mind 287

Chapter 16 A Client-Server State of Mind Fox developers are among the most dedicated group of programmers in the world. Attend a conference and ask how many have been using the product for more than ten years, and you’ll get a large number of hands raised, and probably a few snickers (“Ten? Hell, I’ve been using Fox since it was dBASE II in 1981!”) But with this depth of experience comes a certain mindset in the ways of doing things, and it can be hard to change habits ingrained over decades of practice. It’s time to consider changing that outlook to a client-server state of mind. This chapter covers the conceptual differences and gets you thinking in terms of client-server architecture and what it means to your systems when you work with record sets instead of whole tables.

One of the major influences in building FoxPro applications was Richard Grossman’s landmark demonstration application, Pro-Demo, that shipped with FoxPro 1. 0 in the early ‘90s. It featured a file-card interface, where you could select a table and then navigate through every record in it using , , , and buttons. In the DOS world that FoxPro 1.0 lived in, that’s what those buttons looked like, too – text strings surrounded by less-than and greater-than signs. For more than a decade, Fox developers relied on that paradigm to design their user interfaces. Times have changed, but the interface has not. In some ways, that interface is timeless – witness the page navigation buttons in Adobe Reader or the playback buttons on your DVD player or iPod. What is different, though, is the idea that you can move through the entire table. Back in 1992, when a table frequently had 500, 1,000, or even 5,000 records, that interface was workable. In situations where the table held 10,000, 50,000, or... in some cases, 100,000 records, that interface was still workable, as long as you also had a button to help the user avoid having to click 74,200 times to get to “Slawinski.” When the table has 37,000,000 records and is stored on a database server, you’re just not going to want to – or be able to – have at the entire table in one fell swoop. And here is the crux of a client-server system – your first move is always to select the subset of records you want to look at, in order to narrow down that 37,000,000 to a more manageable number, like, say, 105. Once you do so, many of the problems stay the same – moving from one to the next, changing the order of the records you’re looking at, saving changes, and so on. To be sure, there are still some other differences. For example, while you’re working with just a small subset of parent records, you may be also dealing with the entire set of values in a lookup table, which can introduce some new dynamics. But the fundamental difference between the single user and multi-user LAN based systems you’ve been working with in the past is that you’re now dealing with a small subset of the entire database. Lest you get despondent, though, in bleak anticipation of having to relearn everything in order to build client-server apps, let me reassure you – one of the reasons Visual FoxPro is such an excellent client-server front end is once you bring down a record set from the back end, you can work with it using the VFP data handling commands you already know and love.


This makes VFP much easier to use than those other client-server front ends that have no concept of a ‘record pointer’, and force you to move through a small set of records with arcane syntax and workaround constructs. VFP was built with this type of work in mind!

A bird’s-eye view of a client-server application Let’s go through the Reader’s Digest condensed version of a client-server application, comparing it with a typical multi-user LAN application. First off, we’ll assume there’s a MySQL database ‘out there’ that you can connect to. It has a variety of tables related in all sorts of fashions – one-to-one, parent-child, many-tomany, and so on. Just your typical relational database.

Startup Your system will start up, performing all of the usual checks (disk space, access to resources, etc.) and configuring itself as needed according to the configuration parameters of the application. The system may take login information from the operating system login, or it may require a separate login, or it may not require a login at all. The database login parameters may be determined by the user login, or they might not, with all users of the application using the same database credentials. The choice, again, depends on the specific situation of the application’s environment.

Menu Typical systems will have a menu that offers functions to the user. Menus generally offer one of two types of interfaces. The first is to simply open table-centric forms: “Payments,” “Customers,” “Shipments,” and so on. The second provides action-oriented functions, such as “Enter payments,” “New customer wizard,” and “Group shipments.” Let’s look at each of these separately.

File card forms Table-centric forms typically are displayed as file-cards, where all of the information for the entity in question is displayed on a single form, and the “VCR” buttons allow the user to perform functions. My standard toolbar has had these buttons on it for over a decade: First Previous Next Last

Search List Order

Add Edit Delete

Save Revert

If a particular form had a requirement for a specific action, such as “Split Current Payment,” that button would be placed in another area of the form. See Figure 1.

Done


Figure 1. Standard functions go on the toolbar; form-specific functions are controlled by buttons on the form itself. When a user opens a table-centric form in a traditional application, the form loads up the data needed to populate the form, perhaps using the data environment of the form, and the first record is displayed in the form. The user would use the toolbar to navigate between records and then manipulate the record’s data, delete the record, or add a new record. With a client-server application, however, the form isn’t simply opened with data available in it. Instead, the user needs to select one or more records, or (in nicely designed systems) add a brand new record from scratch from the very beginning. This is done via a filter form, which may be a separate form or one page in a page frame on a catch-all form. See Figure 2 for an example of a filter on a separate form.

Figure 2 . A typical form used to filter records in a client-server application.


Action-oriented forms Menu items on an action-oriented menu might read “Enter payments,” “Add customer,” “Group shipments,” and so on. These menu items open forms that perform specific tasks instead of allowing ad-hoc access to the table or data entity in question, perhaps through a wizard or other procedural guide. In a traditional application, the form opens and is already populated with the relevant data, such as open invoices or pending shipments. In this scenario, the form does some preprocessing before it displays its contents to the user. The same kind of processing could also be performed in a client-server application – logic in the form’s opening code (say, the “init” event) would select a subset of data from the database for the user to work with before displaying.

Connecting to the back end In both cases, whether the user hits ‘Find’ or ‘Add New’ in the filter form, or opens a so-called ‘action-centric’ form, the first thing that needs to be done is establish a connection to the database. There are several different approaches to when this is performed, based on the environment and demands of the users of the application. For now, we’ll assume that you’ll establish a connection when it’s time to actually start accessing the database, as opposed to during application startup.

Selecting a subset of records Once the connection is made, the back-end database is accessed, the appropriate records are selected, and the form is filled with data. These last three steps aren’t quite as simplistic as they sound, of course. Digging data out of the database isn’t as straightforward as with a traditional application. In a LAN application, all you might need to do is use PAYMENTS

or, in a slightly more involved scenario, select * from PAYMENTS, LOOKUP ; where PAYMENTS.iidpmttype = LOOKUP.iidtype ; and empty(dPaid) order by dDue ; into cursor csrOpenPayments

Doing the same thing with a client-server database involves something like the following: text to m.lcStr noshow select * from PAYMENTS, LOOKUP where PAYMENTS.iidpmttype = LOOKUP.iidtype and empty(dPaid) order by dDue endtext m.liSuccess = sqlexec(m.liHandle, m.lcStr, "csrOpenPayments") if m.liSuccess > 0 * continue on else * error handling endif


While this may not seem like a lot of extra work, remember each time you touch the database, you need to wrap your contact with the database with a SQLEXEC(), then test for success, handle the error condition, and so on. It can wear on you. Once a user has chosen a set of criteria to narrow down the result set, you’ll want to do a check to make sure they have provided narrow-enough criteria. If your payments table has 500,000 records in it, criteria of “Amount > $1.00” will probably still return too many records. In this situation, you have two choices. The first choice is to return just the first N records (say, 100), let the user know there are more that qualify, and give them a mechanism to fetch the next 100 (sort of like a grandiose ‘Next’ button). The second possibility is to warn them that the criteria they entered has produced too large of a result set, and they need to narrow it down more. You may want to give them hints about how to be more selective, as this can be a frustrating exercise if they aren’t familiar with the data and thus don’t know how to create a more narrowly focused set of criteria.

Writing back to the database When it comes time to write to the database, there’s another set of issues to deal with. The issues involved vary according to what type of ‘write’ you’re doing – adding new data, editing existing data, or deleting data.

Adding Adding a new record to a single table isn’t too difficult. If you have the field definition for the primary key set up for auto-increment, a simple SQL INSERT is all that’s needed. If you’re inserting records into multiple tables, particularly in a parent-child relationship, you need to capture the parent record’s newly generated primary key for use as a foreign key in the child table, but this programming isn’t much different than doing it with native DBFs and VFP’s own auto-incrementing capability. In fact, it can be even easier, as MySQL has a “last_insert_id()” function that returns the last primary key generated on a table on the current connection. If another user sneaks an insert between the time you did the insert and the time you asked for the last_insert_id(), you’ll still get the primary key of the last record you inserted. More on this in Chapter 19.

Deleting Deleting a record, similarly, isn’t arduous. If you’re deleting a record from a single table and there are no external constraints (such as child records depending on the parent that you’re deleting), a simple SQL DELETE is sufficient. Handling the children of parents that are being deleted can be done either in VFP code, much as you’ve done with native DBFs, or via UPDATE triggers inside MySQL, as described in Chapter 10.

Updating Editing and updating an existing record, on the other hand, can require more care. In short, the SQL UPDATE command is used to send data back to the database, but it’s not as easy as INSERTS and DELETES due to the possibility of multiple users editing the same record(s).


Multi-user issues A couple of reviewers asked early on, “Is a client-server application inherently multi-user?” The answer is that this question involves two separate issues – the physical handling of the database files and the logical potential for collisions when two users are trying to update the same piece of data. The way a client-server application handles the first issue is different than how a traditional application handles it, but they both need to deal with the second issue the same way. When a traditional application accesses a DBF file, either for reading or writing, it’s actually touching the file itself via the operating system. As a result, the programmer has to worry about file locks (such as when reindexing) and record locks (when updating a record). With a client-server database back end, the database engine takes care of reading and writing to the database. The programmer doesn’t have to. Data integrity That said, the programmer does have to consider data integrity issues – what happens when two people are trying to write to the same data element? Let’s look at two examples. Suppose two users are trying to update the address field of a customer. In many traditional applications, this would be done via either GATHER MEMVAR or TABLEUPDATE(). In both cases, the entire row has to be locked in order for the one field to be changed. With a client-server application, a SQL UPDATE command is used instead, and the database engine handles the locking necessary. The question remains: who wins? If Alvin is trying to change the address to 100 Albuquerque and Bob is trying to change it to 200 Buffalo, whoever is last to touch the database is the one whose change sticks. While this could be a problem, it’s possibly a problem in the business process – why are two different addresses being entered for the same record? Either one of the source documents is wrong, or the entity being updated is mistaken (“Oh, you mean there are two muffler/donut shops with the same name?”). A legitimate example of two users trying to update the same field involves the quantity field of an inventory record. Suppose two users are trying to decrement the quantity of the same item as they’re placing orders for the item. In a traditional application, the first user locks the record, updates the field, and releases the lock. The second user has to wait for the first user’s lock to be released before being allowed to issue their own update. There are several ways to approach this issue with a database back end. The first is to use MySQL’s record locking capability, similar to VFP. The second is to wrap the operation in a transaction. And the third is to use a semaphore field to indicate that the record is being edited; releasing the semaphore once the edit has been committed.

Closing the connection Finally, once a database action has been finished, the connection should be closed. This may seem similar to the old xBASE adage of closing tables when they’re not in use, and indeed, in one instance it is – the reasoning is that a connection takes up resources, and you don’t want to leave them lying around, unused but sucking up RAM and bandwidth. As with the connection issue earlier, the ultimate call on when to close the connection depends on your environment and the needs of the users of your application.


Conclusion/Summary In this chapter, I laid out the road map for how a Visual FoxPro client-server application is structured, and how it differs from a traditional LAN application. In the next chapter, I’ll discuss some details in how VFP commands and functions differ between the two architectures, and then we’ll get into building a live application. Updates and corrections to this chapter can be found on Hentzenwerke’s Web site, www.hentzenwerke.com. Click “Catalog” and navigate to the page for this book.


Chapter 17: xBase to SQL Conversion Issues 295

Chapter 17 xBase to SQL Conversion Issues During your career as a client-server developer, you’re bound to run into a situation where a customer or end user asks you to convert a traditional LAN application to work with a SQL back end. Even if the application has been designed with this conversion in mind, it’s not an easy job; at the other end of the scope, such a process can be arduous to the point of requiring a rewrite. As a result, there isn’t a simple cookbook approach, nor a one-size-fits-all strategy. Nonetheless, the tip sheet isn’t completely blank. Here are some issues and reference charts to help you along.

If you’ve ever tried to convert an old xBase-style system to client-server architecture, I’m sure at this point you had to put the book down while doubled over in laughter. “Convert?” you chortle, “You mean ‘Rewrite all over from scratch’, don’t you?” Now that you’ve recovered, take pity on those poor souls who are faced with 20,000 lines of “SKIP FOR X > numX WHILE...”-style code and examine the similarities and differences of Visual FoxPro’s native tools compared with MySQL. There are three broad areas to consider: architecture, commands, and functions. Let’s take a look at each in turn.

NOTE: Some areas of VFP/MySQL aren’t going to change anytime soon. The command and function list is not one of these static areas. Be sure to look for updates and corrections to this chapter as noted at the end of this chapter!

Architecture How a Visual FoxPro application was originally designed has a lot to do with how easily it can be converted. There are generally four categories an application can be put into; each category is a different level of how well the data access was centralized, which in turn determines how easily it will be to convert the data access to a different (client-server) mechanism.

Intermixed xBase The worst of all possible worlds (might as well get the bad news over with first) is where data access is sprinkled through every method in every form via old-style xBase LOCATE, SKIP, SET RELATION TO, and DO WHILE !EOF() programming commands and their ilk. (Probably throw a few @SAYs, RLOCK()s, and ??? output in there at the oddest times, too.) The user interface, business logic, and data access have all been munged into one inseparable mess, and it will likely be a nightmare to convert this tightly bound monolithic application into two separate layers. However, before you start thinking about falling on your corporate sword in despair, let me mention that I’ve seen this situation handled with a resourceful and clever use of remote views. A remote view was created identical to each local table, and the views were substituted into the code for table accesses. There was still some work to be done to handle differences in


data types between the back end and the local tables, but because the original application was written in a very simplistic way, this technique served them fairly well. The end result was still somewhat fragile for a production application, but it was good enough to serve as a respite from a situation where VFP’s native data engine wasn’t robust enough to handle the data set size. If you’re faced with this type of conversion, the first thing you’ll want to do is get a “size and scope” estimate of the application so you can see what you’re up against. Create a list of modules, programs, and/or methods that contain data access code, and do a search for VFP xBase and SQL commands. (There’s a list in the “Command Comparison” section later in this chapter.) Once you identify each place where the database is touched from within the code, go back through and look through all the forms and reports to see if the database is directly referenced in the controls or fields. It may be possible to perform the ‘view switcheroo’ if you have a lot of forms and/or reports that directly access the database – create views that match the tables, and replace the references to the tables with references to the views. As far as handling blocks of old-style xBase code in programs or methods, you’ll have to look at each individual construct to see what is going on and what you can do to perform the conversions. Some suggestions are included in the Command Comparison section.

SQL commands A step up from the Intermixed xBase scenario is when the application was created using SQL commands for all data access. Data access code is still stuffed away in every nook and cranny of the application, but for the most part, you’ll be able to do straightforward replacements of a VFP SQL command with a comparable call to SQLEXEC. You will, however, spend most of your time learning about the compatibilities (and incompatibilities) of VFP and MySQL’s SQL implementations. It is with this scenario in mind that the sections on Command Comparison and Function Cross Reference, included later in this chapter, were originally created.

Central data access mechanism The next best possibility is if the application was built using a centralized data access mechanism. These go by various names, such as ‘data factory’ or ‘data handling object’, and can be implemented either as classes or in programmatic code. But the end result is the same – all access to the VFP data is handled through a centralized clearinghouse, much like major airlines routing all of their airplanes through hubs. Hopefully, though, the application’s data isn’t treated with the same hostile attitude as some of the airlines treat their passengers.... Typically, methods of a data handling object are called each time a form or report needs data from the database as well as when a form or process needs to write data back. Depending on how this data access mechanism was put together, it may need some overhaul in terms of replacing scatter/gather commands or tableupdate() function calls with the appropriate SQL commands. However, since the work is all done in just one place, it should be easier to code and test than if you had to spelunk your way through every program and/or method in the entire application, as you will in the next levels. You still need to do the same work with handling differences between data types, SQL commands, and functions, but all of the work is going to be in one place. Pieces of both the Command Comparison section (to help with SQL commands) and the Function Cross Reference (to help with data types and functions) should be useful.


Built with views Finally, the good news. The best of all possible worlds is if the application was built with views. If they’re local views, all that is required is to create a new set of remote views that map to the existing local views. If they’re remote views using local data, then supposedly it’s even be easier – just swap out one connection for another and you’re all set. Well, supposedly. Unless you’re very, very lucky, there’ll be some tweaking to do of the new views, but the majority of the work should be done for you. There are two types of work that you’ll be faced with. First, you may have to change the views themselves. The new views may not map exactly to the old ones, regardless of whether they’re local or remote. The reason is that the data types aren’t identical, and so you need to make some changes to the view definitions to account for the different data types in the MySQL database versus what you used in the VFP database. The second type of work is going through your VFP code to determine what has to be changed due to the new views having different data types. For instance, if your VFP code is working with a (local or remote) view that assumes a certain field is a logical, you need to change that code to deal with the new MySQL data type, whatever you chose that to be – a CHAR(1) or a BIT(1), or even an ENUM or a numeric.

Command comparison VFP has two types of commands – “new-ish” SQL commands, such as SELECT, INSERT, and UPDATE, and old style xBase commands like LOCATE and SKIP.

SQL commands While both VFP and MySQL support SQL, there are multiple flavors of SQL, and VFP’s implementation has grown over the years. However, in order to keep VFP backward compatible, it is possible to write a completely valid SQL statement in VFP that won’t work in MySQL. “How is this possible?”, you ask. Here’s an example that bites almost every developer at one point or another. The clauses in a VFP SQL statement can be put in nearly any order. Thus, this statement select cnabiz, cnaf, cnal from BIZ, PERSON ; where biz.iidbiz = person.iidbiz ; order by cnabiz, cnal

will produce the same results as select cnabiz, cnaf, cnal from BIZ, PERSON ; order by cnabiz, cnal where biz.iidbiz = person.iidbiz ;

In VFP, that is. Executing the first statement in MySQL generates the same results as in VFP, but the second will not. The second will not run at all, instead preferring to throw an error. In order to figure out what went wrong, the easiest way to test a SQL statement is to use the MySQL Query Browser to run the statement. Pasting the second statement in the Query Browser generates the following error:


You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'where biz.iidbiz = person.iidbiz' at line 2

Yikes! As just shown, if MySQL doesn’t permit the syntax, it will throw an error; most often, the error message displayed will give you a clue about what needs to be corrected. While I’m on the subject of using the Query Browser to test VFP statements, remember that the use of a semi-colon (“;”) as a line continuation character in the Query Browser isn’t allowed; doing so throws its own error. For instance, suppose you paste the following statement in the Query Browser and hit Ctrl-Enter to execute. select cnabiz, cnaf, cnal from BIZ, PERSON ; where biz.iidbiz = person.iidbiz ; order by cnabiz

MySQL will return the following error: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'order by cnabiz' at line 1

The phrase “line 1” might throw you until you realize that MySQL thinks that all three lines are each a complete statement. Since it identifies the ‘order by cnabiz’ clause, it should be easy to quickly spot what went wrong. This will hit you a couple of times before you get used to taking a quick look at the statement, slap your forehead, and say, “Oh, right, semicolons!” Of course, once you have that one figured out, here’s a subtle bug that can show up as a result of “knowing” to simply remove the semi-colons. Suppose you paste the following code into a Query Browser window: select cnabiz, cnaf, cnal from biz, person ; where biz.iidbiz = person.iidbiz order by cnabiz

However, you then move your cursor up to the semi-colon at the end of the first line, intending to delete it, and then someone distracts you for a second. You turn back, forgetting that your goal was to delete the semi-colon, and you hit Ctrl-Enter again... and the statement runs without error, and generates a result set. “Huh?” as you suddenly remember that you were going to get rid of the semi-colon. What happened is that the MySQL Query Browser has run just the first line, ending in “PERSON ;” – and the result set is a Cartesian product of BIZ and PERSON. Hopefully you won’t do that too many times. One other thing to remember is that while VFP will accept SQL statements with four letter abbreviations (such as SELE * FROM customer WHER nAmt > 100), MySQL isn’t interested in handling anything with a VFP-style abbreviation. The error message the Query Browser generates in this situation is easy to understand: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'sele * from cust' at line 1


So if you’re one of those old-fashioned types who got used to four letter abbreviations (like me!), it’s time to change your spots. You’ll have to spell out every keyword.

xBase commands xBase commands are another matter. xBase commands that are positionally oriented won’t work against the back-end. Don’t try sending “GO TOP” to the server, for instance; MySQL will just stare at you. (Actually, it’ll generate an error...) However, once you bring your result set from the back-end as a local cursor with a valid SQL statement, you can ‘skip’, ‘go top’, ‘scan’, ‘locate’, and ‘do while !eof()’ to your heart’s content. With VFP and MySQL, you get the best of both worlds! Similarly, old-style xBase commands for modifying the database – append, scatter/gather, delete – will not work against the back-end. However, again, they will all work fine with the local cursor. You just have to move updated data from the local cursor back to the back-end via SQL INSERT, UPDATE, and DELETE. The following table lists old-style xBase and their comparable SQL commands.

Table 1. xBase and SQL equivalents xBase command

SQL equivalent

locate

select

set relation to

select/join

replace

update

append blank

insert

delete

delete

seek

select

Use

select

Function cross reference As far as functions go, the big thing to remember is that you can’t pass Visual FoxPro functions to MySQL unless MySQL supports the same function – with the same syntax. So a SQL SELECT with a strtran()generates an error when you send it to the back-end. Beyond that, you’re still free to use VFP functions however you want in your code that processes data being fetched from the back-end. Thus, you’ll need to translate VFP functions to their MySQL equivalents. In some cases, there’s a one-to-one correspondence, which makes this job easier. For example, acos() and atan() both work identically in both VFP and MySQL, just in case these had you on the edge of your chair. Others have a close, but not identical, relationship. VFP’s asc() function matches MySQL’s ascii(), while VFP’s iif() works the same as MySQL’s if(). Same functionality, but different spellings. In some cases, the names are different, such as VFP’s alltrim() and MySQL’s trim(). Then there are functions that do sorta the same thing, but with different arguments or somewhat different results. VFP’s inlist() simply tells you whether an item is in a list, while MySQL’s field() tells you which position the item is in (or returns 0 if it’s not found).


As you can tell, then, putting together a cross reference isn’t a simple matter of matching the players on team A with the corresponding players on team B. It’s more like matching positions on a baseball team with positions on a football team. That said, what I did is compile a list of VFP functions that have corresponding functions in MySQL – but only those that have a match. Given that there are over 400 functions in VFP, many do not have any sort of corresponding capability in MySQL; fklabel(), getobject(), and mdown() come to mind. There are a few that one might think would have a MySQL equivalent, such as mdy() and occurs(). Since it’s logical to think there would be a match, I’ve included them in the list. For each VFP function in the list, I identified the corresponding MySQL function. I explain if it’s an exact match, if it’s similar but not identical, how they differ, and, for some, I listed MySQL functions that can be used to perform the same task, albeit in a different way, and perhaps in conjunction with other MySQL functions. Theoretically, just about every VFP function could be replaced by some arcane combination of MySQL functions, commands, and system calls. At some point, I drew an arbitrary line and said, “Enough is enough.” Finally, there are a few functions in MySQL that don’t have a match in VFP; I listed those at the end of each section, just to alert you to the extra capabilities in MySQL. I also want to point out that the purpose of this section is to provide a starting point for finding equivalent MySQL functions. I have not duplicated the complete syntax for every possible option and variety for every MySQL function – that’s what the online documentation is for! In other words, this is NOT the Hacker’s Guide to MySQL Functions!

Interactive evaluation of MySQL functions By the way, you’re used to using the VFP Command Window to evaluate functions, like so: ? strtran("Star Trek", "Trek", "Wars") Star Wars

But MySQL doesn’t have a “Command Window,” so how can you experiment with MySQL functions? Use the Query Browser. Enter a function as an argument to a SELECT statement, like so select now()

and execute the command. The return value is displayed in the Results Pane. Although this doesn’t give you quite the same instantaneous feedback as the VFP Command Window, and it doesn’t produce the same hierarchical listing as multiple commands executed in the Command Window one after another, you can still see the results from multiple functions by including them as sequential arguments to the SELECT command, like so: select now(), dayname(now()), monthname(now())

and as shown in Figure 1.


Figure 1. Displaying the results of multiple functions in a single SELECT command.

Math functions This section includes the usual complement of trig functions (examples not provided because, in the 20 years of using xBase languages, I needed to use a trig function exactly once) and other mathematical/numerically oriented operations. VFP

MySQL

Description

abs()

abs()

Returns absolute value. Functions are identical. VFP: abs(-42) =: 42 MySQL: abs(-43) =: 43

acos()

acos()

Returns arc cosine. Functions are identical.

asin()

asin()

Returns arc sine. Functions are identical.

atan()

atan()

Returns arc tangent. Functions are identical.

atn2()

atan2()

Returns arc tangent of two variables. Functions are identical.

ceiling()

ceiling()

Returns next highest integer. Functions are identical. VFP: ceiling(123.56) =: 124 MySQL: ceiling(123.56) =: 124

cos()

cos()

Returns cosine. Functions are identical.

dtor()

radians()

Converts degrees to radians. Functions are identical.


VFP

MySQL

Description

exp()

ln()

VFP's exp() returns e^x where x is passed as an argument. MySQL's ln() performs the opposite, returning x when the value of e^x is passed as an argument. VFP: exp(0.6931471805599501) =: 2.000000000 MySQL: ln(2) =: 0.6931471805599501

floor()

floor()

Returns integral portion of a decimal value. VFP: floor(123.56) =: 123 MySQL: floor(123.56) =: 123

int()

floor() truncate()

Returns integral portion of a decimal value. MySQL's truncate() allows truncating of a decimal value at positions other than the decimal. VFP: =: MySQL: =: MySQL: =: MySQL: =: MySQL: =:

log()

log() ln()

Returns the natural log (base 'e') of an argument. MySQL's log() function can be passed a second argument to operate on a non-e base (e.g. base 10). VFP: =: MySQL: =: MySQL: =: MySQL: =:

log10()

log10() log2()

int(123.56) 123 floor(123.56) 123 truncate(123.56,0) 123 truncate(123.56,1) 123.5 truncate(123.56,-1) 120

log(10) 2.302585 ln(10) 2.302585 log(10) 2.302585 log(10,100) 2

Returns the base 10 log of an argument. Both functions are identical. MySQL's log2() returns the base 2 log of an argument. VFP: log10(1000) =: 3 MySQL: log10(1000) =: 3 MySQL: log2(8) =: 3

mod()

mod()

Returns the remainder after dividing the first argument by the second. Functions are identical. Handy when splitting a dozen donuts between five kids to determine how many Dad ends up with. VFP: mod(12,5) =: 2 MySQL: mod(12,5) =: 2

pi()

pi()

Returns pi. VFP returns 15 places, MySQL returns 16. If this difference is important to your application, I'd be interested in hearing more. VFP: pi()*1.000000000000000000


VFP

MySQL

Description =: 3.141592653589793000 MySQL: pi()*1.000000000000000000 =: 3.141592653589793100

rand()

rand()

Returns a random number between 0 and 1. VFP: rand() =: 0.248349792834 MySQL: rand() =: 0.839842778371

round

round() truncate()

Rounds a number, to the given number of decimal places. If not given a second argument, MySQL rounds to the nearest integer. See truncate() in the VFP int() function entry. VFP: round(1059.26,1) =: 1059.3 MySQL: round(1059.26,1) =: 1059.3 MySQL: round(1059.26) =: 1059

rtod()

degrees()

Converts radians to degrees. Functions are identical.

sin()

sin()

Returns sine. Functions are identical.

sqrt()

sqrt() power()

Returns square root of argument. MySQL's power() is equivalent to VFP's x^n syntax. VFP: =: VFP: =: MySQL: =: MySQL: =:

tan()

tan()

sqrt(9) 3 2^3 8 sqrt(9) 3 power(2,3) 8

Returns tangent. Functions are identical.

Lists, comparison, logic functions These are a smörgåsbord of functions that involve a list or that compare two or more values. VFP

MySQL

Description

between()

between..and interval()

The between comparison operator is not a function, per se, but it provides the same capability. It returns a 1 or a 0 as true/false operators. In addition, MySQL's interval() can accomplish something similar with a bit of creativity. interval() allows the comparison of a series of values and returns the index of the first match where n is less than n+1. VFP: =: MySQL: =: MySQL: =: MySQL: =:

icase()

case

between(42, 10, 100) .t. 42 between 10 and 100 1 interval(10,42) 0 interval(42,100) 0

Creates a VFP-style CASE statement logic structure in a single line. VFP's icase() evaluates arguments until it finds one that is true, and then returns


VFP

MySQL

Description the next argument, while MySQL's case function uses an initial argument and then arguments to pairs of when/then keywords. VFP: type='c1' =: icase(type='a', 'A', type='b', 'B', type='c1', 'C1', type='zap', 'None') =: C1 MySQL: case 44 when 'a' then 'A' when 'b' then 'BB' else 'NOTHING' end =: NOTHING MySQL: case 'b' when 'a' then 'A' when 'b' then 'BB' else 'NOTHING' end =: BB

iif()

if()

Immediate if – compares two items and returns the second or third value in the function. VFP: iif(1>2, 'a', 'b') =: b MySQL: if(1>2, 'a', 'b') =: b

inlist()

in() field() elt() find_in_set() coalesce()

in() is the closest MySQL operator to VFP's inlist(). It returns 1 if an item is in a list. VFP: inlist(999,1,2,3,4,5,6,7,8,999,10,11,12) =: .t. MySQL: 999 in (1,2,3,4,5,6,7,8,999,10,11,12) =: 1

VFP's inlist() function is rather limited in terms of identifying where in a list an item is located. By comparison, MySQL's field() function not only identifies whether or not an item is in a list, but in which position. MySQL: field(999,1,2,3,4,5,6,7,8,999,10,11,12) =: 9 MySQL: field(42,1,2,3,4,5,6,7,8,999,10,11,12) =: 0

The related elt() function performs the opposite operation on a list – returns the Nth item in a list when passed N as an initial argument. MySQL: elt(6, 'a','b','c','d','e','F','g','h','i','j') =: F

The find_in_set() works similarly to field() except that the items in the set do not have to be delimited individually. MySQL: find_in_set(999,'1,2,3,4,5,6,7,8,999,10,11,12') =: 9

Finally, coalesce() returns the first non-NULL value in a list. MySQL: coalesce(NULL, NULL, 1/0, NULL, 42, NULL, 999) =: 42

max()

greatest()

Returns the largest value in a list. VFP: max(10, 42, 100) =: 100 MySQL: greatest(10, 42, 100) =: 100

min()

least()

Returns the smallest value in a list. VFP: min(10, 42, 100) : 10 MySQL: least(10, 42, 100) : 10


Empty/Blank/NULL functions A variety of functions that deal with empty, blank, and NULL data situations. VFP

MySQL

Description

evl()

Short for "Empty VaLue" (along the lines of nvl().) It performs a substitution on an empty value. No equivalent in MySQL.

empty()

Evaluates whether an expression is empty. No equivalent in MySQL.

isblank()

Evaluates whether an expression is blank. No equivalent in MySQL.

isnull()

ifnull() nullif()

NULL evaluation. The MySQL implementation of ifnull() takes some careful attention. If the first argument is null, the second argument is returned, else (if the first argument is not null), the first argument is returned back. VFP: m.i_am_null = .null. : isnull(m.i_am_null) =: .t. VFP: m.unknown = 10/0 :isnull(m.unknown) =: .f. MySQL: ifnull("I am not a null", "second") =: I am not a null MySQL: ifnull(NULL, "We have a null winner!") =: We have a null winner! MySQL: ifnull(1/0, "1/0 is null") =: 1/0 is null

A related function is MySQL's nullif(), which returns NULL if both arguments are equal, and the first argument if they are not equal. MySQL: nullif(10,10) =: NULL MySQL: nullif("Visual FoxPro", "MySQL") =: Visual FoxPro

nvl()

Short for "Null VaLue" – no equivalent in MySQL.

Data/database functions A variety of functions that deal with, gasp, data. VFP

MySQL

Description

dbc()

database()

Returns the name of the (VFP) current or (MySQL) default database. If none, returns an empty string (VFP) or NULL (MySQL).

dbused()

database()

In VFP, returns whether or not the database specified as an argument is open (whether or not it's selected). In MySQL, either a database is the default (used) or not.

fdate()

There is no equivalent to VFP's fdate() function in MySQL. All of the 'timestamp' type of functions in MySQL have to do with current system time or values in a table, regardless of when the table was actually last touched.

ftime()

There is no equivalent to VFP's ftime() function in MySQL. All of the 'timestamp' type of functions in MySQL have to do with current system time or values in a table, regardless of when the table was actually last touched.

flock()

get_lock()

In VFP, flock() locks an entire table. In MySQL, get_lock() simulates


VFP

getautoincvalue()

MySQL

Description

release_lock()

application or record locks. release_lock() releases the most recent lock. Locks are different than VFP locks; see the MySQL documentation for more information.

last_insert_id() In VFP, getautoincvalue() returns the last AutoInc value – during the current session. In MySQL, last_insert_id() returns the first automatically generated value by the most recently executed INSERT or UPDATE statement. See the MySQL documentation for the many nuances of last_insert_id(). VFP: getautoincvalue() =: 195 MySQL: last_insert_id() =: 195

lupdate()

reccount()

There is no equivalent to VFP's lupdate() function in MySQL. All of the 'timestamp' type of functions in MySQL have to do with current system time or values in a table, regardless of when the table was actually last touched. found_rows() row_count()

There is no direct equivalent to reccount() in MySQL. However, row_count() produces the same result as the _tally variable – displaying the number of rows that were inserted, updated or deleted by the most recent SQL statement. VFP: : =: MySQL: : =:

select * from cust where cState = 'WI' _tally 207 select * from cust where cState = 'WI' row_count() 207

In addition, found_rows() (in concert with the SQL_CALC_FOUND_ROWS expression) will return the number of rows that would be returned by a SQL command without applying a LIMIT clause. recno() rlock()

There is no equivalent to VFP's recno() function in MySQL. get_lock() release_lock()

In VFP, rlock() locks the current record. In MySQL, get_lock() simulates application or record locks. release_lock() releases the most recent lock. MySQL locks are different than VFP locks; see the MySQL documentation for more information.

Three interesting data-related MySQL functions without a direct match in VFP are default(), name_const(), and values(). default() returns the default value for a column – similar to what you could do with VFP’s dbgetprop()– while name_const() can be used to specifically name a column in a result set (something like the ‘AS’ keyword in SQL SELECT). values() can be used in an UPDATE part of a SQL statement to refer to column values in the INSERT part of the same statement. See the MySQL documentation for more details and examples on all three of these.


String functions VFP

MySQL

Description

+

concat() concat_ws()

Concatenates strings, optionally with separators.

alltrim

trim()

VFP: "Visual" + " " + "FoxPro" =: Visual FoxPro MySQL: concat('Visual', 'Fox', 'Pro') =: VisualFoxPro MySQL: concat_ws('!','Visual','Fox','Pro') =: Visual!Fox!Pro

Trims specified character from both ends of a string. VFP: =: VFP: =: MySQL: =: MySQL: =:

asc()

ascii() ord()

allt(" no spaces wanted here! ") no spaces wanted here! allt("AAABBBCCCBBBAAA", 1, "A") BBBCCCBBB trim(" no spaces wanted here! ") no spaces wanted here! trim(both 'A' from 'AAABBBCCCBBBAAA') BBBCCCBBB

Returns the ASCII code for a character. ord() works with multi-byte characters. VFP: asc('a') =: 97 MySQL: ascii('b') =: 98

at(), at_c(), atc(), atcc()

instr() locate() position()

Locates one string within another – instr ("in string") returns the position of one string within another, locate is the same as 'instr' except with the arguments reversed, and position is the same as 'locate' except with different syntax. VFP: =: MySQL: =: MySQL: =: MySQL: =:

at("bob", "100 bobolink way") 5 instr("bob", "100 bobolink way") 5 locate("100 bobolink way", "bob") 5 position("bob" in "100 bobolink way") 5

atline(), atcline()

See "Full Text Search Functions" topic in MySQL help file.

chr()

No equivalent.

chrtran(), chrtranc()

insert() replace() quote()

insert() substitutes a string inside another, with a specified start location and length to replace, while replace() just grabs every instance of string X and replaces it with string Y. quote() is a specialized string substitution function that handles escaping. VFP: =: MySQL: =: MySQL =: MySQL: =:

filetostr()

load_file()

chrtran('HarrisonFrankCarrie', 'Frank', 'YODA') HDOOisonYODACDOOie insert('RoundAnyHouse', 6, 3, 'The') RoundTheHouse replace('oconomowoc', 'o', 'O') OcOnOmOwOc quote('Don\'t!!!') 'Don\'t'

Reads in a file and converts it to a string


VFP

MySQL

Description VFP: filetostr("c:\config.fpw") =: set path to \dev_utils MySQL: update CUST set NOTES=LOAD_FILE('c:\config.fpw') =: notes field contains c:\config.fpw contents

isalpha()

No equivalent.

islower()

No equivalent.

isupper()

No equivalent.

left(), leftc()

left()

len(), lenc()

char_length()

lower()

lower() lcase()

ltrim

Returns the N most characters from the left end. VFP: left("Visual FoxPro",8) =: Visual F MySQL: left("Visual FoxPro",8) =: Visual F

Returns the length of a character string VFP: len('moretext') =: 8 MySQL: char_length('moretext') =: 8

ltrim() trim()

occurs()

Converts a string to all lower case characters. VFP: lower("Visual FoxPro and MySQL") =: visual foxpro and mysql MySQL: lower("Visual FoxPro and MySQL") =: visual foxpro and mysql MySQL: lcase("Visual FoxPro and MySQL") =: visual foxpro and mysql

Trims specified character from left end of a string VFP: ltrim('ZZZAAAZZZ', 1, 'Z') =: AAAZZZ MySQL: ltrim(' ABCXYZ') =: ABCXYZ MySQL: trim(leading 'Z' from 'ZZZAAAZZZ') =: AAAZZZ

No equivalent.

padc()

lpad(), rpad()

See VFP's padl() and padr()

padl()

lpad()

Extends a string to a specified length, using a specified character to fill characters in from the left. VFP: padl("herman", 10, "O") =: OOOOherman MySQL: lpad('herman', 10, 'O') =: OOOOherman

padr()

rpad()

Extends a string to a specified length, using a specified character to fill characters in from the right. VFP: padl("herman", 10, "O") =: hermanOOOO MySQL: rpad('herman', 10, 'O') =: hermanOOOO

proper()

No equivalent.

rat(), ratc()

See VFP's at(), atc().


VFP

MySQL

ratline() replicate()

Description See "Full Text Search Functions" topic in MySQL help file.

repeat()

Repeats a character string a specified number of times. VFP: repl("Mo_", 3) =: Mo_Mo_Mo_ MySQL: repeat("Mo_", 3) =: Mo_Mo_Mo_

right(), rightc()

right()

rtrim

rtrim() trim

space()

Returns the N most characters from the right end. VFP: right("Visual FoxPro",3) =: Pro MySQL: right("Visual FoxPro",3) =: Pro

space()

Trims specified character from right end of a string VFP: rtrim("MMMoooMMM", 1, "M") =: MMMooo MySQL: rtrim("Whinney ") + "!" =: Whinney! MySQL: trim(trailing 'M' from 'MMMoooMMM') =: MMMooo

Returns a character string of a specified number of spaces. VFP: "!" + space(10) + "!" =: ! ! MySQL: "!" + space(10) + "!" =: ! !

str(), strconv()

No equivalent.

strextract()

No equivalent.

strtofile()

No equivalent.

strtran()

insert() replace() quote()

insert() substitutes a string inside another, with a specified start location and length to replace, while replace() just grabs every instance of string X and replaces it with string Y. quote() is a specialized string substitution function that handles escaping. VFP: =: MySQL: =: MySQL =: MySQL: =:

strtran('HarrisonFrankCarrie', 'Frank', 'YODA') HarrisonYODACarrie insert('RoundAnyHouse', 6, 3, 'The') RoundTheHouse replace('oconomowoc', 'o', 'O') OcOnOmOwOc quote('Don\'t!!!') 'Don\'t'

VFP: stuff("DarthVader", 6, 5, "Daddy") =: DarthDaddy MySQL: insert('RoundAnyHouse', 6, 3, 'The') =: RoundTheHouse

stuff(), stuffc()

insert()

substr()

Returns part of another string. mid() and substring() are identical. mid() substring_index() allows you to specify a length in terms of occurrences of substring() substring_index() a specified character, either from the left or right. VFP: substr("Visual FoxPro", 3, 6) =: sual F MySQL: mid("Visual FoxPro", 3, 6) =: sual F MySQL: substring("Visual FoxPro", 3, 6)


VFP

MySQL

Description =: sual F MySQL: substring_index('oconomowoc', 'o', 3) =: ocon MySQL: substring_index('oconomowoc', 'o', -2) =: woc

trim()

trim()

See examples for VFP's trim().

upper()

upper() ucase()

Converts a string to all lower case characters. upper() and ucase() are identical. VFP: upper("Visual FoxPro and MySQL") =: VISUAL FOXPRO AND MYSQL MySQL: upper("Visual FoxPro and MySQL") =: VISUAL FOXPRO AND MYSQL MySQL: ucase("Visual FoxPro and MySQL") =: VISUAL FOXPRO AND MYSQL

Two interesting string-related MySQL functions without a direct match in VFP are char() and reverse(). char() creates a text string out of the characters of ASCII codes passed in as arguments. char(48,57,65,90,97,122)

returns 09AZaz

while reverse() takes a string and reverses it for the result, like so: reverse("Every good boy does fine")

returns enif seod yob doog yreve

Many a parlor trick can be had with these two!

Date functions MySQL has a huge number of date/time functions. While it is possible, with enough ingenuity, to morph most any VFP date/time function into something that resembles a MySQL function, and vice-versa, these are the functions that have a reasonably close match. Additional date functions in MySQL that you won’t find matches for in VFP are listed below the table. Enjoy. VFP

MySQL

Description

{}

date() makedate() maketime()

Returns date or datetime value. See MySQL documentation for details on formats. VFP: {^2006/12/31} =: 2006-12-31


VFP

MySQL

Description VFP: =: MySQL: =: MySQL: =: MySQL: =:

cdow

dayname()

{^2006/12/31 23:47:59} 2006-12-31 23:47:59 date('2006-12-31') 2006-12-31 makedate(2006,365) 2006-12-31 maketime(23,47,59) 23:47:59

Returns day of week (Monday...) VFP: cdow({^2006/12/31}) =: Sunday MySQL: dayname(date('2006-12-31')) =: Sunday

cmonth()

monthname()

Returns month of year (January...) VFP: cmonth({^2006-12-31}) =: December MySQL: monthname(date('2006-12-31')) =: December

ctod() ctot()

str_to_date()

Converts a date in the form of a string, together with a descriptive format string, to a date/datetime value. VFP: =: VFP: =: MySQL: =: MySQL: =: MySQL: =: MySQL: =:

date()

datetime()

day()

curdate() current_date()

now() current_timest amp() sysdate()

day() dayofmonth()

ctod('2006/12/31') 12/31/2006 ctot('23:45:59') 1899/12/30 23:45:59 str_to_date('12/31/2006', '%m/%d/%Y') 2006.12.31 str_to_date('2006/12/31 23 22 49', '%Y/%m/%d %H %i %s') 2006-12-31 23:22:49 str_to_date('2006/12/31', '%Y/%m/%d') 2006-12-31 str_to_date('23:22:49', '%H:%i:%s') 23:22:49

Returns current date. current_date() is a synonym for curdate(). VFP: date() =: 12/31/2006 MySQL: curdate() =: 12/31/2006

Returns current datetime. now() and sysdate() are slightly different, see the MySQL documentation for a detailed explanation. current_timestamp() is a synonym for now(). VFP: datetime() =: 12/31/2006 23:47:59 MySQL: now() =: 2006-12-31 23:47:59 MySQL: sysdate() =: 2006-12-31 23:47:59

Returns numeric day of month. day() and dayofmonth() are identical. VFP: day({^2006/12/31}) =: 31 MySQL: day(date('2006-12-31')) =: 31 MySQL: dayofmonth(date('2006-12-31'))


VFP

MySQL

Description =:

dmy() dow()

31

No equivalent. dayofweek() weekday()

Returns numeric day of week. Indices differ. VFP, Sunday=1, Monday=2.... MySQL dayofweek(): Sunday=1, Monday=2... MySQL weekday(): Monday=0, Tuesday=1... VFP: dow(date()) =: 2 MySQL: dayofweek(now()) =: 2 MySQL: weekday(now()) =: 0

dtoc() dtos()

date_format()

dtot()

time_format()

Converts a date value to a string via user-provided format specifiers. VFP: dtoc({^2006/12/31}) =: 2006/12/31 MySQL: date_format(date('2006-12-31'),'%Y %m %d') =: 2006 12 31

Converts a time value to a string via user-provided format specifiers. VFP: dtot({^2006-12-31}) =: 2006/12/31 12:00:00 AM MySQL: time_format(now(), '%H %i %s') =: 13 59 26

gomonth()

hour()

period_add() period_diff() last_day() date_add() date_sub() adddate() subdate() addtime() subtime()

hour()

period_add() adds months to a period in the format of CCYYMM, while period_diff() returns the number of months between two periods. MySQL's last_day() function performs the same function as VFP's gomonth() function does when gomonth() is passed the last day of a month. Use MySQL's adddate() and subdate() functions in concert with last_day() to produce gomonth(). date_add() and date_sub() are synonyms for adddate() and subdate(). See addtime() and subtime() also. VFP: =: MySQL: =: MySQL: =: MySQL: =: MySQL:

gomonth({^2006-12-31},2) 2007-2-28 period_add(200612, 2) 200702 last_day('2006/12/14') 2006-12-31 adddate('2006/12/14', INTERVAL 2 month) 2007-02-14 last_day(adddate('2006/12/14', INTERVAL 2 month)) =: 2007-2-28

Returns hour portion of date/datetime expression VFP: hour({^2006-12-31 23:47:59}) =: 23 MySQL: hour(time('23:47:59')) =: 23

mdy() minute

No equivalent. minute()

Returns minute portion of date/datetime expression VFP: minute({^2006-12-31 23:47:59}) =: 47 MySQL: minute(time('23:47:59')) =: 47

month()

month()

Returns month portion of date/datetime expression VFP: month({^2006-12-31 23:47:59})


VFP

MySQL

Description =: 12 MySQL: month(time('23:47:59')) =: 47

quarter()

quarter()

st

th

Returns the quarter of the year (1 through 4 ) of a given date. Identical in VFP and MySQL. VFP: quarter({^2006-12-31 23:47:59}) =: 4 MySQL: quarter('2006/12/31') =: 4

sec()

second()

Returns seconds portion of date/datetime expression VFP: sec({^2006-12-31 23:47:59}) =: 59 MySQL: seconds(time('23:47:59')) =: 59

seconds()

microsecond() time_to_sec()

Seconds since midnight (millisecond resolution)

sys(1)

from_days() to_days()

System Date in Julian Day #

sys(2)

time_to_sec()

Seconds since midnight (integer)

sys(10)


Character date from Julian Day #

sys(11)


Julian Day # from date expression

time()

curtime() current_time()

Returns current time. current_time() is a synonym for curtime().

time()

time()

VFP: time() =: 23:47:59 MySQL: curtime() =: 23:47:59

Extracts time portion of datetime value. VFP's version extracts to hundredths of a second. VFP: time(datetime()) =: 23:47:59.01 MySQL: time(now()) =: 23:47:59

ttoc()

No direct equivalent.

ttod()

No direct equivalent.

week()

week() weekofyear() yearweek()

Returns week of the year. VFP's week() takes two arguments that control which week starts the year. MySQL's week() takes a second argument that controls where the week begins while weekofyear() is equivalent to week() with a second argument of '3'. See the respective doc files for all the nuances. yearweek() returns the year and the week. VFP: =: VFP: =: MySQL: =: MySQL: =: MySQL:

week({^2006-12-31}) 1 week({^2006-12-31}, 1, 2) 53 week(date('2006-12-31'),0) 52 week(date('2006-12-31'),3) 1 yearweek(date('2006-12-31'))


VFP

MySQL

Description =:

year()

year()

53

Returns year portion of date expression. VFP: year({^2006-12-31}) =: 2006 MySQL: year(date('206-12-31')) =: 2006

MySQL date functions without a direct match in VFP that I think are interesting include the following: datediff(d1, d2): returns the number of days between two dates. dayofyear(d1): returns the day of the year (from 1 to 366). extract(unit from d1): returns one or more segments of a date, based on the ‘unit’ passed in as an argument. Differs from, say, year() or month() in that multiple segments can be returned, e.g. extract(year_month from ‘2006-12-31’) returns 200612. timediff(t1, t2): returns the number of hours/minutes/seconds between two times.

System information functions This, too, is a smörgåsbord of functions that dig up information about the ‘system’. VFP

MySQL

Description

id()

current_user() user() connection_id() uuid()

VFP's id() function returns machine name and user id. The MySQL functions return the username and hostname; they may all be the same but don't have to be. current_user() is the account actually used to authenticate the current client. user() is the account specified during login. system_user() and session_user() are synonyms for user(). VFP: id() =: MORDOR # frodo MySQL: current_user() =: frodo@MORDOR

The MysQL function, connection_id(), returns the connection ID (thread ID) for the connection. This is not the VFP handle. MySQL: connection_id() =: 47388

As an aside, MySQL's uuid() returns a Universal Unique Identifier (UUID), which is a 128-bit number of the form aaaaaaaa-bbbb-cccc-ddddeeeeeeeeeeee where each piece is a hexidecimal number. MySQL: uuid() =: 021495c3-c16e-1029-bbda-bb9cdb78c36c

version()

version()

VFP's version() function returns the version of VFP while MySQL's version() function returns the version of MySQL. Go figure. VFP: version() =: Visual FoxPro 09.00.0000.3504 for Windows MySQL: version() =: 5.0.21-community-nt

Interesting system information MySQL functions without a direct match in VFP include the following two IP address-related functions. inet_aton() takes as an argument a dotted-quad IP address and returns the numeric version of the address, while inet_ntoa does the reverse:


inet_aton('1.2.3.4')

returns 16909060

while inet_ntoa(1234567890)

returns 73.150.2.210

Fun stuff, and handy in lots of situations.

Miscellaneous functions Finally, we come to the “didn’t fit anywhere else so we jammed them in here” category. This list includes functions to deal with soundex and code pages. VFP

MySQL

Description

difference()

soundex()

difference() returns an integer that represents how similar two strings are. The closest that MySQL has to this is the soundex() function. See the entry for VFP's soundex().

like() likec()

soundex()

like() compares two expressions, including wildcards. The closest that MySQL has is the soundex() function.

soundex()

soundex()

Returns a value calculated from the soundex algorithm that can be used to determine how similar two strings are. Note that the values returned by VFP and MySQL for identical strings are not always the same because they use different versions of the algorithm. VFP: soundex("herman") =: H655 MySQL: soundex("herman") =: H650

cpconvert() cpcurrent() cpdbf()

charset() collation()

cpconvert(), cpcurrent(), and cpdbf() all provide code page functionality. MySQL's charset() function returns the character set (e.g. latin1, utf8, etc.) of a string argument, while collation() returns the collation (e.g. latin1_swedish_ci, utf8_general_ci, etc.) of a string argument.

Conclusion/Summary While there is no magic bullet that will provide you with a successful conversion to a clientserver architecture, having a list of gotchas and the proper tools in hand to assist in the effort can go a long way. In this chapter we covered potential issues with architecture, compared SQL and xBase commands, and then went through MySQL’s function library to find matches for the VFP functions you might encounter in your applications. Now it’s time to build the components you want in a client-server application – from scratch!


Updates and corrections to this chapter can be found on Hentzenwerke’s Web site, www.hentzenwerke.com. Click “Catalog” and navigate to the page for this book.

Chapter 18: A Client-Server User Interface for Querying 317

Chapter 18 A Client-Server User Interface for Querying Up to now, this discussion has been all programmatic. What about forms? You know – those things that the user looks at data in? In this chapter, I’ll cover the creation of a typical user interface for querying a database, made up of a filter form for specifying parameters, and a results form that displays the matching records and allows the user to navigate among them. I’ll show both a simple parent-child join as well as a form that uses a couple of lookup (‘minor-entity’) tables.

The purpose of this chapter is to create a mechanism for read-only operations – essentially, queries. In the next chapter, I’ll cover writing operations – add/edit/delete. As I mentioned in Chapter 16, your mind-set for accessing a client/server back end is different than the approach used with the traditional LAN application. In the olden xBase days, you designed your form as if you had access to the entire database (because, indeed, you did.) I always likened it to opening up a file cabinet with a huge drawer of file cards, and allowing the user to flip from the first to the second, the third, the fourth, and so on, back and forth, through the whole drawer. Instead, you need to decide which records you want, bring those into your form, and provide navigation tools for that small subset. To carry the “file cabinet full of file cards” analogy forward, you’ll be grabbing just a handful of those cards (copies, actually) and putting them on your desk to work with. A word of warning when you’re designing your UI; many developers use a small test data set. The reason is because they probably don’t have a realistic test database that contains millions of records and because, if they make a mistake (say, where the query DOES try to bring down the entire back end), they don’t want to be waiting until it finishes. Quite annoying. The problem with this small data set is a mistake that causes you to hit the entire database won’t really be noticed. Keep in the back of your mind, “What if I had 37,000,000 records in here?” Use a small data set to perform functional testing, but then do some performance testing against a larger dataset. When your UI will handle those 37 million records with pretty much the same alacrity as the 105 that you have in your test database, then things are set up properly.

The logical and physical components Back to the design itself. Your UI consists of two logical pieces – one that provides userentered criteria (“filter parameters”) to select the desired records, and a second that allows the user to work with the result set. There are many ways to implement this in pr actice, and the differences really boil down to personal preference. Some people like to use a page frame, with the first page holding filter controls and the second (and, perhaps, subsequent) pages displaying the results. The second page might show a list of matches via a grid, and the third page would display the details of the record highlighted in the second page’s grid.


Another mechanism is to use two separate forms. The first form would accept filter parameters from the user. Entering a value (or values) then causes that form to close and open a second form that displays the results. This form might use a page frame with separate pages for results and details, similar to the first mechanism, or it might simply cram everything – results list and details – on the same form. This last design is the one I prefer, and will use in this chapter’s examples. As there are a lot of things going on here, I’m going to create this mechanism in a series of steps, each example building on the shoulders of the previous one. First, I’ll create a single ‘results’ form that demonstrates the single page with both the results list and the detail fields crammed onto it. This form will use a local database as its datasource so we can concentrate on the design of the form, populating the results list, and then showing how navigating from one record to the next will update the controls for the detail fields. The second version will bring some complexity into play, showing how to display child records as you navigate from one record to the next in the results list. The third version will add the filter criteria form to the mix. The filter form will only have one or two simple text fields; now isn’t the time to build a complex query builder interface. The parameters entered into the filter form will be passed to the results form that will then winnow down the local database and create a smaller results set, mimicking the performance of the client-server mechanism we’ll build in a moment. The big difference between this example and the previous one is that we’re producing a small result set that the results form will work with. Once you’re comfortable with the structure of this two form interface, we’ll swap out the code that handles the local data and replace it with code that connects to the MySQL back end and returns selected data. Since the operation of the rest of the form is based on the results set, this will simply be a matter of changing the method that populates the results set! When you open the source code for the forms in this chapter (as well as subsequent chapters), you’ll see there isn’t a lot of sophisticated subclassing, with objects flying all over the place. You’re trying to learn how to build a UI for client/server, and see what the code underneath looks like, so that’s what we’re going to focus on. It can be terribly distracting if you’re not too experienced with objects (and, based on the reviewer population, a significant percentage are coming up to speed on the object-oriented capabilities of VFP at the same time that they’re learning client-server), to the point that including that into the mix would be more confusing than helpful. For example, a lot of folks like providing an independent toolbar for functions to control objects on a data entry form, such as navigation, add/edit/delete, search, and so on. I’m one of them. But the code that incorporates a separate toolbar with the form involves communication between the toolbar and the form – no, wait, make that the active form – determining which toolbar buttons need to be available... you see where I’m going with this. Incorporating all of this toolbar code with your first query form just gets in the way of what we’re trying to do, so I’ll put all the necessary buttons right on the form. Same thing for separate ‘biz objects’ and n-tier programming models. That’s too much to handle for the developer who is working with their first client-server system. We’ll pass on these items for the time being as well. Once you’re done with this book and comfortable with client-server philosophies and practices, consider picking up one of the frameworks that consists of a robust object-oriented set of data handling classes as part of the package if you want more sophistication. The purpose of this book is to provide a simple enough set of code to teach the principles of client-


server development without a robust infrastructure. The purpose of such a framework is to provide that infrastructure.

Source code The source code for this chapter is contained in two files, common.zip and ch18.zip. You can use these files in one of two ways. The first is to create a directory, like ‘\mysqlbook\ch18’, and unzip both files into this directory. Then open VFP and change the default directory to the directory you unzipped the files into, like so: cd \mysqlbook\ch18

When you open the forms, you’ll probably be prompted to look for the hwctrl.vcx (and other) class libraries. Just point to the directory you put them in. The second way is to create separate directories for common files and the source code for this chapter, like so: e:\mysqlbook\common e:\mysqlbook\ch18

Then run VFP, change the default directory to \mysqlbook\ch18, and set your path to include the common directory: cd \mysqlbook\ch18 m.lcOrigPath = set('path') m.lcNewPath = m.lcOrigPath + iif(empty(m.lcOrigPath), "", ",") ; + "\mysqlbook\common" set path to (m.lcNewPath)

The common.zip file contains program files that are used in more than one chapter’s examples e.g., procedure files, base classes, and graphics. The chapter file contains all of the program files and database files needed for the examples in this chapter.

Base classes (hwctrl.vcx) All of the forms in these examples are built using the same set of base classes found in hwctrl.vcx. These classes included a sub-classed version of the visual VFP controls we’ll need for these examples, and a set of form classes. These subclasses range from very simple, with only a couple of custom properties and methods each, to fairly simple, with the bare minimum of properties and methods needed to get the job done. Here is the nickel tour of the classes we’re going to use for this chapter’s examples. The forms that display results are all based on the hwfrmnav class. This class is based on the hwfrm form class, sub-classed from the VFP form base class. While the hwfrm class has some new methods and properties, none of them are germane to this specific example, so I’ll bypass a detailed explanation. However, the hwfrmnav class has some very important modifications. First, it has several controls that are included by default. The Done button automatically calls the form’s Close() method. The two textboxes with yellow backgrounds are what I call


‘developer-only’ controls. They’re only displayed on the form when the form is run interactively (in the VFP IDE), or when a special ‘run-as-developer’ parameter is passed to the application during test or production use. I typically use these ‘developer-only’ controls to display information such as primary key values or other pieces of data used during testing or debugging. The listbox on the hwfrmnav class comes from the hwlstnav class, and has two very important items – a custom property and a custom method. The aItems[1] property is used to populate the list; RowSourceType is set to 5-Array, and RowSource is set to this.aItems. Thus, in order to fill the listbox, aItems just needs to be populated. The custom AnyChange method is called from both the InteractiveChange and ProgrammaticChange methods of the native class. All code that is run when the list is used is put in AnyChange, instead of having to duplicate it in both the IAC and PC methods.

Simple navigation form (simplenav.scx) The first version of the form, simplenav.scx, contains just a listbox used for navigation and standard controls to display data. Its purpose is to display the businesses and locations in the INS database, and allow the user to navigate from one to another, much like they would in the aforementioned file cabinet by scrolling up and down through the listbox. See Figure 1.

Figure 1. A simple form using a listbox as a navigation tool. This form is based on a supporting cursor named csrRes that contains the result set that will populate the controls in the form. This cursor can be created from a single table, or from the join of multiple tables. The important point is that the result set consists of a limited number of records. Identifying fields from the supporting data set are loaded into the listbox, allowing the user to scroll through the table and display all fields for the record in the controls on the right side of the form. Under the hood, the init() method calls GetResults() that creates the csrRes cursor. In this first incarnation, GetResults() simply runs a SQL SELECT to create a join of


businesses and locations, and then populates the listbox’s aItems array with a user-friendly list of business names. The AnyChange method of the listbox contains the following code select csrRes go this.ListIndex thisform.TableToForm(this.ListIndex)

Since there is a one-to-one relation between the listbox’s array and the supporting cursor, any action in the listbox, be it interactive or programmatic, will cause the record pointer in the csrRes cursor to be moved to the record that matches the selected array element. If you’re thinking that you wouldn’t want to use this interface to navigate through a cursor that had 37,000,000 records, you’re right, both technically and practically. Technically, versions of VFP before 9.0 can’t support an array that big – it’s limited to 65,000 elements in an array, and even with ‘just’ that many, it’d likely be slow going. (VFP 9 supports two GB.) In a practical sense, how would you scroll through a listbox that contains thousands and thousands of rows? This is why we’re going to use a filter to winnow down the number of records in the result set. Once the record pointer is moved, the form’s custom TableToForm method is called with the record number of the newly selected record. TableToForm is a centralized location for all the code that is involved in displaying the record’s data in the form’s controls. (If you’re wondering why this wasn’t done by simply setting the ControlSource of each control to the underlying cursor, the answer is that in this simple example, it could have been. However, we’re shortly going to run into situations where the mapping won’t be as clean as one field to one control. Thus, we introduce this mechanism now, and will expand on it in subsequent examples.)

Simple navigation with children form (simplenav_withchild.scx) The second version of the form, simplenav_withchild.scx, expands on the first example by displaying child records for the business-location results set in a pair of listboxes. The first listbox shows the contacts for a specific location – voice and fax phone numbers, website URL, and so on. The second listbox shows the categories that the business belongs to. It is a listbox, as some organizations can be grouped under more than one listing. See Figure 2.


Figure 2 . A simple form with child records displayed in listboxes. This form has several changes. The obvious ones are a larger form area and the addition of two listboxes for the child records. Under the hood, the TableToForm method has been enhanced to populate both listboxes. (Told you we were going to need it!) Upon moving to a new record in the result set, the TableToForm method grabs the associated child records based on the primary key in the csrRes cursor, and populates the listboxes with the results.

Simple navigation with filter (simplenav_withchild_andfilter1/2.scx) Now it’s time to skinny down that list of records in the result set. We do this with a separate filter form that collects the parameter and passes it on to the results form. See Figure 3.

Figure 3 . The Filter form allows the user to enter a value to select a subset of records. Once a filter value has been entered and the user selects Find, the filter form is closed and the results form is opened. However, contrary to what you might think, the filter value isn’t


passed as a parameter. Instead, an object reference to the calling form is passed. In the GetResults() method, this object reference is used to determine the value of the filter value. While it’s true that it would be easier to simply pass the name of the business in this example, what happens when you have three, four, five, or more filter values that have to be passed to the results form? Passing a whole string of parameters, some of which may not exist, quickly becomes a mess; it’s much easier and cleaner to pass a single object reference whose utility will grow as the filter form gets more complex. And applications always get more complex!

Note: Andy Kramek interjects here: Once you’re comfortable with this technique of passing an object reference instead of a string of parameters, consider the use of a parameter object, where each property’s name is the name of a parameter, and the values in the properties are the values to be passed. This technique avoids the tight-coupling of these two forms (what happens when the name of one of the controls on the filter form changes? Chaos!), uses fewer system resources, and provides better encapsulation. Now onto the results form. The layout of the form doesn’t have to change at all; indeed, nothing changes except the contents of two methods. The init() method of the results form now includes a line to accept the filter form’s object reference parameter: thisform.oCallingForm = toCaller

The GetResults() method has two changes. The first is the determination of the filter value on the filter form: m.lcBusinessName = alltrim(thisform.oCallingForm.hwtxtBusinessName.value)

and the second is a modification to the SQL SELECT statement, so the filter value is actually used in the query: select biz.iidbiz, biz.cnabiz, ; loc.iidloc, loc.cstreet, loc.ccity, loc.cstate, loc.czip, lIsHQ, tClosed ; from BIZ, LOC ; where BIZ.iidbiz = LOC.iidbiz ; and BIZ.cNaBiz like m.lcBusinessName + "%" ; order by biz.cNaBiz, loc.cStreet ; into curs csrRes

I know I just said that the UI of the form doesn’t have to change, but if you looked at the new example in the source code, you’ll see a new control – a read-only text box positioned above the list box. See Figure 4 if you haven’t opened the source code yet.


Figure 4 . The results form now includes a read-only textbox that displays the Filter criteria. This control will contain the filter expression used to populate the result set. I think it’s a courtesy to the user to show them what expression was used to produce the results they’re looking at. In the GetResults() method, I grabbed the value of the label of the filter value text box and appended the value entered in the text box to create the filter expression. I used the value of the label instead of the field name, figuring that “Business Name” was easier for the user to understand than, say, “cNaBiz”. Run the form and enter a single letter of the alphabet, such as “m” or “a.” The listbox in the result form displays just those matching records. Or try entering a full name, like “BaskinRobbins” or “Pizza Hut.” The last change made to this form is in the close() method. The filter form is closed, meaning it’s no longer visible. However, the results form still has a reference to it. To show this, open the Locals debug window, and then run the filter form. You’ll see an object variable named “simplenav_withchild_andfilter1:” if you click on the plus sign to the left, the properties of the object displays. When you eventually close the results form, you’ll still see the variable “simplenav_withchild_andfilter1” displayed in the Locals window. This is a ‘dangling reference’ to an object that’s been destroyed, and this can be bad; in the worlds of IBM in the days of Lotus 1-2-3 and the original IBM PC, “unexpected results may occur.” Here, those results usually consist of C5 memory errors at the most inopportune times. To get rid of the reference, add the following code release simplenav_withchild_andfilter1 dodefault()

to the Close() method of the results form.


Simple navigation with MySQL back end (simplenav_withmysql1/2.scx) Now it’s time for the main event – hooking up our filter and results forms to talk to MySQL instead of local DBFs. Hold on tight! In this example, we use the same INS MySQL database that we used in earlier chapters; if you haven’t set it up yet, you need to now if you want to follow along with the source code for this chapter. (See Chapter 12 for instructions about creating the INS database.) There are four separate changes that need to be made to the simplenav_withchild_andfilter forms. (1) a connection to the database needs to be made (and disconnected when the form is closed), (2) the resulting handle needs to be stored as part of the form, (3) the GetResults() method needs to get its results from the back-end instead of from local DBFs, and finally, (4) the TableToForm() method needs to be updated to handle child SELECTS. Because we’re connecting to the INS database via a connection string, we need to add one more change that allows the host, username, and password to be passed to the filter form via parameters, just like in examples in earlier chapters. Thus, we call the filter form like this: do form simplenav_withmysql1 with 'localhost', 'bob', 'secret'

but change the parameter values as appropriate for your system. The filter form looks the same as the one shown in Figure 4, but the results form has been slightly modified as shown in Figure 5.

Figure 5 . The results form now includes the handle of the database connection in the title.


If you’re scratching your head, trying to figure out what the difference between Figures 4 and 5 are, look in the title bar of the form.

Pass connection parameters to simplenav_withmysql1 First, we need to add three properties to this filter form. These properties hold the values that are passed into the form so they can be accessed from the results form. The results form is where we’re making actual connection to the database, but we enter the ‘system’ from the filter form, so that’s where we need to send in the connection parameters. The properties are named cdbhost, cdbun, and cdbpw. All character strings, all database related, all initialized to be empty strings. Next, we need to modify the init() method of the form to accept parameters, and to store them to the form’s properties: lparameters m.tcdbhost, m.tcdbUN, m.tcdbPW if pcount() < 3 messagebox("You must pass the host, username & password as a parameter:" ; + chr(10) + chr(13) + chr(10) + chr(13) ; + "do FORM with 'localhost', 'bob', 'secret'") return .f. endif thisform.cdbhost = m.tcdbhost thisform.cdbUN = m.tcdbUN thisform.cdbPW = m.tcdbPW dodefault()

Notice that the init() returns false if less than three parameters are sent in; this prevents the form from instantiating. No sense in teasing the user, is there? Finally, the Find button’s click() method has been tweaked to call the new results form: do form simplenav_withmysql2 with thisform

Now let’s look at the results form, simplenav_withmysql2.scx.

Store connection handle in simplenav_withmysql2 Once we create a connection, it needs to be stored as a property of the form so it can be referred to during various database operations. The property, iHandle, is this property. It’s initialized to be a numeric value.

simplenav_withmysql2.init() The init() method of the results form needs to set up the connection before any work with the database itself is performed. Init() looks like this: thisform.cdbhost = alltrim(thisform.oCallingForm.cdbhost) thisform.cdbUN = alltrim(thisform.oCallingForm.cdbUN) thisform.cdbPW = alltrim(thisform.oCallingForm.cdbPW) if thisform.CreateConnection() if !thisform.GetResults() return .f. endif else


return .f. endif

Since the database work starts with the GetResults() method, init()needs to call the CreateConnection() method just before the GetResults() method. If CreateConnection() succeeds, GetResults() is called next, and if that also succeeds, init()’s job is finished. Now, what if the attempt to create the connection fails? We need to provide feedback and not open the results form – what would be the point? If the connection fails, we have the CreateConnection() method return a .f. back to its caller, the form’s init() method. The init() method didn’t simply just call CreateConnection() blindly. Instead, it tests the return value from the method, and if .f. is returned, init()in turn returns .f., which prevents the form from instantiating. The CreateConnection() method has already displayed an error dialog to the user, so init() doesn’t have to perform that chore. GetResults() works through the same mechanism. If GetResults() fails (we’ll see the code in a moment), GetResults() also returns a .f. value, which causes init() to return a .f., and, again, the results form does not instantiate. As an aside, the definition of failure of GetResults() would be due to bad SQL statements, for example, or a back-end database problem – an empty result set is not a failure.

CreateConnection() The CreateConnection() method looks like this: m.liH=sqlstringconnect( ; + "DRIVER={MySQL ODBC 3.51 Driver};" ; + "SERVER="+thisform.cdbhost+";UID="+thisform.cdbUN ; +";PWD="+thisform.cdbPW+";database=INS") if m.liH < 1 messagebox("No connection; handle is:"+transform(m.liH)+":") return .f. else thisform.iHandle = m.liH endif thisform.Caption = alltrim(thisform.Caption) ; + " (Handle: " + transform(thisform.iHandle) + ")"

The last line in this code snippet shows the modification of the caption in the title bar to reflect the handle.

GetResults() Let’s look at the changes made to GetResults() now. First, the way you build search expressions to send to MySQL is different. You need to add a “%” to the end of the filter expression, like so: m.lcBusinessName = alltrim(thisform.oCallingForm.hwtxtBusinessName.value) + "%"

and then use ‘like’ in the SQL SELECT command. We’re going to use the “text to m.lcStr” command to build the m.lcStr string that’s going to be sent to MySQL via SQLEXEC(). The SQL SELECT command itself also changes, because the fields in the MySQL LOC table are different from the DBF version.


text to m.lcStr textmerge noshow pretext 15 select biz.iidbiz, biz.cnabiz, loc.iidloc, loc.cno, loc.edir, loc.cstreet, loc.csuf, loc.csecline, loc.ccity, loc.cstate, loc.czip, iishq, tclosed from BIZ, LOC where BIZ.iidbiz = LOC.iidbiz and BIZ.cNaBiz like '<>' order by biz.cNaBiz, loc.cStreet endtext

Next, there needs to be two levels of trapping in the GetResults() method, as opposed to just one when we were dealing with local DBFs. The first level of trapping concerns whether the SQLEXEC() function actually worked or not. The following code snippet shows that if SQLEXEC() doesn’t work, the error is recorded via aerror(), the user is notified, and then GetResults() returns .f. This sends the function back to the init(), which in turn returns a .f., thus preventing the results form from opening. The pseudo-code looks like this: m.liX = sqlexec(thisform.iHandle, m.lcStr, "csrRes") if m.liX > 0 * code for success else * record error with aerror * notify user with messagebox() * then return .f. endif

The details of the ELSE branch of the IF construct look like this: local aOops[1] m.liHowManyErrors = aerror(aOops) for m.li = 1 to m.liHowManyErrors insert into ZOOPS ; (inoerr, ctext, ctextodbc, csqlstate, inoerrsql, ihandle, tadded) ; values ; (aOops[m.li,1], aOops[m.li,2], aOops[m.li,3], aOops[m.li,4], ; aOops[m.li,5], aOops[m.li,6], datetime()) next * display nothing messagebox("SQL Execution failed; code is" ; + chr(10) + chr(13) + chr(10) + chr(13) ; + transform(m.lcStr)) return .f.

The use of the aerror() function and the recording of the errors isn’t anything new – it’s nearly verbatim the same error trapping code shown in Chapter 12. However, after the error(s) have been logged, note the messagebox that notifies the user about what happened, gives them the string of code that failed, and then returns .f. to the calling function. An alternative to displaying the code to the user is recording it in the ZOOPS error logging table. The second level, built into the ‘success’ side of the ‘m.liX > 0’ test, determines whether or not any result sets have been returned. If > 0, at least one result set has been returned, so check to see if there are any records in it, and proceed from there. if m.liX > 0


* display if used('csrRes') and reccount('csrRes') > 0 m.liNumRows = reccount("csrRes") * fill array populating the listbox else * no records in csrRes * display nothing in array endif

The empty csrRes cursor is a valid occurrence, so we don’t want to return .f. from the ELSE branch of the IF construct. For example, if the user entered “ZYX-o-plenty,” it would be reasonable to assume there might not be any matches in the INS database. Well, at least around here, there aren’t any businesses with that name. That takes care of the GetResults() method. Now we turn our attention to the TableToForm() method.

TableToForm() The TableToForm() method is where we populate the detail controls on the Results form. There are two places where changes have to be made. The first is where individual controls are assigned values from fields in csrRes. The fields in the MySQL LOC table don’t map directly to the LOC DBF fields – the MySQL structure has individual attributes of a street address broken up into separate fields. Thus, those attributes have to be reassembled to be displayed in the ‘Street’ textbox. Trivial, but needs to be done: thisform.hwtxtStreet.Value ; = alltrim(csrRes.cno) + " " ; + alltrim(csrRes.edir) + " " ; + alltrim(csrRes.cStreet) + " " ; + alltrim(csrRes.cSuf) + " " ; + iif(empty(csrRes.cSecLine), "", ", ") ; + alltrim(csrRes.cSecLine)

The second change is more involved, not conceptually, but in a busy-work kind of way. The child listboxes, for Contacts (populated from the COOR table) and for Business Types (populated from the ZLOOKUP table via the BIZCAT join table) need to be filled with data from the MySQL tables. This means more SQLEXEC() functions using modified versions of the SQL SELECTS that were used with local DBFs. The code to dig out the contacts for the business highlighted in the Business listbox on the left of the Results form looks like this: text to m.lcStr textmerge noshow select iidcoor, cdata, etype from COOR where coor.iidloc = <> endtext m.liX = sqlexec(thisform.iHandle, m.lcStr, "csrResCoor")

If the SQLEXEC() function succeeds, the Contacts list box is populated by using the following code: if m.liX > 0


m.liNumRows = reccount("csrResCoor") if m.liNumRows > 0 dimension thisform.hwlstCoor.aItems[m.liNumRows,2] m.li = 1 scan thisform.hwlstCoor.aItems[m.li,1] ; = csrResCoor.eType + " " + csrResCoor.cData thisform.hwlstCoor.aItems[m.li,2] = csrResCoor.iidCoor endscan else * code to populate the list box if there were no COOR records endif

There’s one subtle difference in the line of code that fills the first column of the hwlstCoor.aItems array. In the local DBF version – Simple Navigation with Children – the first column is created by simply concatenating the contact type (such as ‘voice’) and the contact data (such as the phone number) like so: thisform.hwlstcoor.aItems[m.lii,1] = aCoor[m.lii,3] + aCoor[m.lii,2]

This produces a list box that looks like this: voice fax web

414.555.1212 414.555.1213 www.example.com

because the contact type data is stored in a fixed length field, which means there will always be padding out to the full length of the cType field. However, note that the type of contact in the MySQL COOR table is an enumerated field, which means there isn’t any padding in the data in MySQL. Thus, the VFP cursor field representing the enumerated field data will be as wide as the longest field value, with no padding. The listbox would look like this: voice414.555.1212 fax 414.555.1213 web www.example.com

So we’ll need to add a space, like so: thisform.hwlstCoor.aItems[m.li,1] = csrResCoor.eType + " " + csrResCoor.cData

Note also that the field name has changed from cType in the DBF to eType in MySQL. The rest of the code in the TableToForm() method is straightforward, featuring the same error trapping for failure of SQLEXEC() as well as testing for whether the resulting cursor has no records.

Close() The last change in the simplenav_withmysql2.scx form has to do with closing the form. We want to make sure to close the connection. This is done like so: m.liResult = sqldisconnect(thisform.iHandle)


debugout iif(m.liResult = 1, ; "Successful disconnect", "Unsuccessful disconnect") release simplenav_withmysql1 dodefault()

You may notice that I have a debugout statement indicating success or failure – it doesn’t take any appreciable amount of time, but can be handy while debugging during interactive development.

Where do you create connections? The creation of the connection to the back end is one of the major differences between local DBF access and using a SQL database. The code to create the connection could be placed in any number of places besides the Results form. The Filter form is one possible location; during startup of the application that hosts the forms would be another; and a completely separate method or object that is called whenever it is needed by whatever needs it would be a third. How do you decide? As you saw, I have it in the Results form, and perhaps you’re wondering why. The answer is that it was expedient to do so for this set of examples, and provided a demonstration of one method. The argument here is to keep the connection tightly bound to the code that’s going to use it: open it, use it, close it. Keeping connections open when they’re not being used can be a bad thing for several reasons. This is a throwback to the old FoxBase days when people opened a table, worked with it, and closed it as soon as they were done with it – both because of the limited number of tables that could be open at a single time as well as the corruption problems that occasionally showed up when unused tables were left open. Some Xbase developers may also remember the license fees for the multi-user versions of their products. Client-server systems have similarities to these issues. First, a connection requires resources on the server. In systems where there are a lot of concurrent users, it is possible to bog down a server if each user opens and maintains a unique connection during their entire session. A client-server application with 30 users probably won’t face this issue; one with 20,000 users very well might. A second reason is licenses. Client-server applications will sometimes use the “shut down the connection as soon as possible” mechanism due to licensing issues. If the back-end database requires license fees for each connection, then someone is probably paying attention to the number of connections available, and “minimizing concurrent connections” becomes a checklist item on the specification. While file corruption isn’t the same problem for client-server applications as it was in the early days of multi-user LAN applications on fragile networks, having large numbers of unused connections on an underpowered server can cause problems or instability. So if you need to do the open-use-shut thing, the question is where? I didn’t put it in the filter form because doing so didn’t fit in with this “minimalist connection” paradigm: the user might open the Filter form, and then sit on it, keeping it open for minutes, hours, or days. If they do so, there’s an unused connection floating around. Thus, the mechanism used was to create the connection at the most low-level, reasonable place – the form that was actually going to use the connection. That’s not to say this is the penultimate rule of connection creation location (did I just say that?), indeed, it isn’t. There are compelling arguments for using a different technique –


creating a connection object independent of the forms that all the forms can latch on to when needed. Let’s look at these reasons. First of all, having a number of open, but unused, connections may not be that much of a drain of resources these days. Compare a few unused connections (one per user) to the potential load on your system if users are continually opening and closing forms (and thus, opening and closing connections). The work involved in the establishment of a connection requires greater resources – obtaining the rights to use the port, locate the server, send a request to connect, supply and validate the credentials, perform the login, and return the results (successful or unsuccessful login) to the client. Thus, the argument continues, establishing a single connection upon application startup, and using a timer to logout if the connection is idle for a pre-determined period of time, is the way to go. Second, it can be time-consuming to open a connection during the instantiation of a form; you may have noticed a slight drag while running even these simple examples, and that’s if you’re running on a local server. If your database server is on another machine, the work creating the connection will have to fight with all the other traffic on the network. Having an already open connection to piggy-back on can be much swifter. Third, if you’re hard-coding the connection information in each form, you have to change each form when the user account changes or the database moves to a different server. That’s obviously inefficient, and you probably put all of the credentials in a centralized place – but then that adds to the work the form has to do – fetching the credentials in order to log in and create the connection. Similarly, if the connection to the server goes down, you need to detect it in all open forms if the form is responsible for creating the connection. If you centralize the connection management, then only one place is responsible for handling problems. However, as I said at the beginning, this is one school of thought. The environment you’re working in – the horsepower and resources of the server, the network configuration and capacity, the number of and work habits of your users, and their demands for responsiveness inside the application all have to be weighed against each other. In short, “it depends” and “your mileage may vary.” Most of the reviewers use the centralized connection management option, and as systems get bigger, this technique proves to be on the winning side. You choose. Just make sure that whichever technique you use, create the framework to track what is happening so you’re not simply guessing about whether you’re running out of resources and that you provide enough flexibility to make changes in a centralized location.

Trade-offs for child and minor entity relationships The location of connection creation isn’t the only design issue involving trade-offs. Another involves handling child and minor entity tables. A child table is a table that contains one or more records related to the record in question; the classic example is invoices – one header record is related to one or more child detail records. A minor entity table is one that contains lookup values, such as a list of states and provinces, or a set of allowed payment codes, or a group of business types. The problem, generically stated, is “when do you go to the server to populate the form from the child or minor entity tables?”


Child tables We’ve seen one incarnation of this problem with the Contacts and Business Types child tables. As you move from one business to the next, the Contacts and Business Types list boxes have to be repopulated. You have essentially two choices. The first is to grab all child records along with the parents in a single command. Then, with the resulting cursor denormalized, perform additional work to garner a unique list of business names to populate the Business list box, and relate that new unique list to the original denormalized cursor that contains all child records. This becomes more difficult if there are multiple sets of child records for each parent record, like our current example does. This also has the danger of showing out-of-date data. If this form was open for a while, the data shown on the form may not be current, since scrolling through the list to another business would show the data that was pulled down when the form was originally opened – and that data may not be the same data that exists on the server right now. The second choice is to do what was shown in this example; grab just the parent records, and as the user navigates from one parent to the other, make additional calls against the server to grab the needed child records for the new parent. The trade-off here is the effort involved in working with a denormalized cursor, which could be sizable, depending on the number of child records, and complex, depending on how many child tables are related to the parent versus the performance hit encountered when having to go back to the server each time a new parent is selected.

Minor entities Another incarnation of this problem is when the available choices for a field come from a minor entity (lookup) table. Suppose you’re working with the aforementioned Invoice table, with child records representing detail lines for a specific invoice. There could easily be a halfdozen minor entity tables involved – state/province, zip code, payment type, payment status, invoice type, sales rep involved, customer service rep involved, shipping method, and so on. Each of these would display in a control that supports multiple fixed choices, such as an option group, a series of checkboxes, a combobox or a listbox. The issue is “when are these controls populated from the server?” Do they get filled from the lookup table on the server once – when the application is started up? Or each time a form that uses them is loaded? Or are they refreshed every time the user moves to a new record, just in case new choices have become available in the last few blimptoseconds? Obviously, loading the minor entity tables into the application’s memory space (a series of ‘global’ arrays, say) just once when the application is started up will speed up the rest of the application, as the back-end doesn’t have to get touched for these data items. Navigation through a result will also be faster, as the navigation form only has to look at data in memory, as opposed to going to disk – the server’s disk. Whether this work is needed depends on the network in question – small tables, few users, modern network – you could well get away with retrieving the lookups from the server all the time. However, that list of product codes – 11,000 to date – is a serious drag on the aging network out in the factory, and caching it locally may well be the ticket to happy users. On the flip side, though, it’s possible that the source tables could be updated – either new records added or existing records changed or deleted – and the user is now looking at outdated lookup values. If those 11,000 product codes are updated once a quarter, caching them locally works. If they’re updated several times a day, then caching in global arrays isn’t going to work.


Alternatively, the minor entities could be refreshed from the server every time a new parent record is selected. The data will be as current as is technically possible, but at the cost of performance of the form, as the user will have to wait for the queries to the back-end to complete. The deciding factors for this situation are twofold. First, how often do the minor entity tables get updated and second, how critical is it that every new value is available instantly? For example, caching the list of North American states and provinces is a pretty safe bet – that list rarely changes. On the other hand, the available sizes and shapes of a hot-selling item on an on-line store could change minute-by-minute. The second factor is performance. In some scenarios, the performance hit may not be significant – it may not even be noticeable. In others, every split second may count. In some applications, a middle ground is chosen – the application loads a set of local tables with lookup values that are then used through the day, saving memory resources while at the same time, not burdening the server with countless hits against tables that rarely change. Ted Roche suggests another potential solution: Store the “date last updated” for each minor entity table. Keep a copy locally and a copy on the server (which is updated automatically in a trigger when a lookup value is updated). Only if that record shows a change should you bother with the bigger query on the main table. As with the connection issue earlier, the ultimate call depends on your environment and the needs of the users of your application.

Conclusion/Summary The user interface is where the rubber meets the road in an application, and an interface that isn’t adjusted for the requirements of a client-server architecture will bring the application to a grinding halt. Once you become comfortable with the mindset of working with just a few records, however, the client-server paradigm makes sense; soon, you’ll wonder how you ever did it any other way. Updates and corrections to this chapter can be found on Hentzenwerke’s Web site, www.hentzenwerke.com. Click “Catalog” and navigate to the page for this book.

Chapter 19: A Client-Server User Interface for Add/Edit/Delete 335

Chapter 19 A Client-Server User Interface for Add/Edit/Delete In the last chapter we finally got into some code. Unfortunately, it was all read-only code. What about ENTERING and SAVING data? Or ADDING records? Maybe even getting rid of them! Sure, you probably want to save data once in a while. Remember the work you went through 15 years ago to learn how to make your applications multi-user – when you couldn’t just assume that only one user was touching the data? A similar paradigm shift is staring at you now as well – except that you already have some experience with handling contention. In this chapter, I’ll discuss the basic mechanism to add, edit, and delete data both in single and multiple table forms. I’ll also cover a number of techniques that are useful during the implementation of these forms.

The purpose of this chapter is to create a mechanism for simple writing operations – add/edit/delete. We’re going to build separate forms to demonstrate adding, editing, and deleting. Then we’ll build a fourth form that puts all three functions together, and throws in the handling of multiple tables as well. First, though, we need to do some housekeeping with the functions that will support these forms.

Creating a class library for client-server functions Until now, we used a procedure file, z_sql.prg, for common functions. Let’s move those into a simple class library. The hwlib.vcx class library already contains the hwlib class. This class has a number of general purpose functions, such as a function that reads INI files. We’re going to add a second class called “cslib” that holds functions specific to client-server applications. First, create another class in our hwlib class library like so: create class cslib of \common\hwlib.vcx as custom

Then add three methods: • • •

z_tfed (test for empty date) z_es (escape string) z_sqlerror (insert SQL error info into ZOOPS)

The class library is instantiated, say, as an object called ocslib, like so: ocslib = newobj("cslib", "hwlib.vcx")

Then, calls to these functions will look like this ocslib.z_es("some string")


or ocslib.z_tfed(a date)

Since this class will only be used with client-server forms, we’ll instantiate the class inside the form. Where? If we were going to bind controls to data, we could do this in the Load() method so the class is available as controls are instantiated. On the other hand, if we’re going to need to pass parameters to the class, this would need to be done in the Init(). We’ll put it in the Load(). Details to come. If you’re following along by running the samples provided in this chapter’s source code files, I suggest you open your Locals window so you can watch variables, including object references, appear and vanish when the forms are created and destroyed.

Building a simple form to add records (ui_add.scx) As this is our first form, it will be very simple – a couple of text boxes and a couple of Save buttons. The purpose is to add records to the BIZ table – a perfect choice since it only has two fields of interest – cNaBiz and cSecLine. The completed form is shown in Figure 1.

Figure 1. A simple form for adding new records.

User interface The user calls this form by passing the hostname, username, and password, like so: do form ui_add with 'localhost', 'whil', 'secret'

and the cursor is positioned in the “Business Name” text box. Once the user enters a Business Name and leaves the field (probably by tabbing out), the “Save and Close” and “Save and Add Another” buttons are enabled. The first button commits


the values in the two controls to a new record in the table and closes the form, while the second adds the record, blanks the text boxes, and then repositions the cursor to the Business Name text box so the user can quickly add another record. The “Done” button acts as a “Just Kidding” release for those times when the user opens the form but then doesn’t actually want to add a record. Because this is an example, this doesn’t do any fancy validation such as searching for duplicate business names or attempting to correct spelling or capitalization of the data.

Internal design Under the hood, this form is based on the hwfrm class. If you’re building this yourself, create the form like so: create form ui_add as hwfrm from hwctrl.vcx

Then add the properties ih and ocslib to the form, and include the code for the Init() method listed shortly. The Init() method is overridden with the code that accepts the hostname, username, and password as parameters and attempts to connect to the database with them. If it’s not successful, the user is notified and the form is not instantiated; if successful, the connection handle is assigned to a property of the form. Next the cslib class is instantiated and a reference to the ocslib object is assigned to a form property as well. Finally, the caption is modified to reflect the connection handle – this last is purely for our purposes during development. * ui_add.init() lparameters m.tcdbhost, m.tcdbUN, m.tcdbPW debugout this.Name + '.Init' if pcount() < 3 messagebox("You must pass the host, username & password as parms, like so:" ; + chr(10) + chr(13) + chr(10) + chr(13) ; + "do FORM with 'localhost', 'bob','secret'") return .f. endif m.liH=sqlstringconnect( ; + "DRIVER={MySQL ODBC 3.51 Driver};" ; + "SERVER="+m.tcdbhost+";UID="+m.tcdbUN+";PWD="+m.tcdbPW+";database=INS") if m.liH < 1 messagebox("No connection; handle is:"+transform(m.liH)+":") return .f. else thisform.iH = m.liH endif dodefault() ocslib = newobject("cslib", "hwlib.vcx") thisform.ocslib = ocslib thisform.Caption = alltrim(thisform.Caption) + " (Handle: " + transform(thisform.iH) + ")" thisform.hwlblSaveFailed.Caption = ''

You can run the form, ui_add, as is even before adding any controls. You’ll get a blank form because there are no controls on it. However, if you have the Locals window open, you’ll


see the olib and ui_add objects created. olib comes from the Init() of the hwfrm class. If you drill into the ui_add object, you’ll see the ocslib object reference. When you close the form, you’ll see both objects destroyed, because the form is no longer in scope. In order to clean things up, add the following code to the close() method: * ui_add.close() m.liResult = sqldisconnect(thisform.iH) debugout iif(m.liResult = 1, "Successful disconnect", "Unsuccessful disconnect") dodefault() release ui_add

This code makes sure the connection is closed and releases the reference created to the form itself. Try closing the form and watching the Locals window, before and after adding this code to the close() method. Now let’s add controls to the ui_add form. Text boxes named hwtxtBusinessName and hwtxtSecondLine, command buttons named hwcmdSaveAndClose, hwcmdSaveAndAddAnother, and hwcmdDone, and a label named hwlblSaveFailed. The LostFocus method for both text boxes calls the custom validate_data() method: * ui_add.validate_data() if empty(thisform.hwtxtBusinessName.value) thisform.hwcmdSaveAndClose.Enabled = .f. thisform.hwcmdSaveAndAddAnother.Enabled = .f. else thisform.hwcmdSaveAndClose.Enabled = .t. thisform.hwcmdSaveAndAddAnother.Enabled = .t. endif

validate_data() is what ensures that there is data in the Business Name text box before the Save command buttons are enabled.

Saving data The Click() method in both Save buttons runs code that calls the form’s Save() method, and then traps for success or failure in the method’s return value. The two methods are different because one shuts down the form while the other clears out the text boxes, disables the Save buttons after a successful save, and repositions the cursor in preparation for another entry. * ui_add.hwcmdSaveAndAddAnother.click() with thisform if .save() .hwtxtBusinessName.Value = "" .hwtxtSecondLine.Value = "" .hwtxtBusinessName.SetFocus() else .hwlblSaveFailed.caption = "An error has happened during save. Please see the error log." .hwcmdsaveAndAddAnother.Enabled = .f. .hwcmdsaveandClose.Enabled = .f. endif endwith


The form’s Save() method, while we’re talking about it, is fairly straightforward for this simple form, but it’s a good place to get our feet wet. * ui_add.save() * save data * iidbiz is auto-incr field in INS.biz m.lcNaBiz = thisform.ocslib.z_es(alltrim(thisform.hwtxtBusinessName.Value)) m.lcNaSec = thisform.ocslib.z_es(alltrim(thisform.hwtxtSecondLine.value)) text to m.lcStr textmerge noshow pretext 7 insert into INS.biz (cnabiz, cnasec, cadded, tadded, cchanged, tchanged) values ('<>', '<>', 'bob', now(), 'bob', now() ) endtext m.liX = sqlexec(thisform.iH, m.lcStr) if m.liX > 0 * continue on return .t. else local aOops[1] m.liHowManyErrors=AERROR(aOops) =thisform.ocslib.z_sqlerror(@aOops, m.lcStr, m.liHowManyErrors) return .f. endif

After sending the contents of both text boxes through our z_es() escape string function, the resulting values are used in the new record being added via a SQL INSERT command. A point worth mentioning, because it’d be easy to miss, is that the primary key for the BIZ table is auto-incremented on the server’s end, so our INSERT command doesn’t need to specify the iidbiz field at all. From our work in previous chapters, you should recognize the “text to ... textmerge” command as well as the << >> delimiters. Errors are saved up via the z_sqlerror() call if the SQLEXEC() function call fails. If you want to play around with this a bit more, try adding validation of the Business Name, ensuring that it is unique, before committing it to the table.

A simple two table edit form (ui_edit.scx) Each example we do gets a bit more complex. This edit example brings a second table into the mix. It provides the ability to navigate through a list of businesses and their locations (using the form created in Chapter 18), and allows edits (and saves!) to both the business info and the locations. For this example, we’re going to ignore the BizTypes, Coordinates, and People child list boxes. Figure 2 shows what our finished form looks like.


Figure 2 . A simple form for editing existing records.

User interface Even a simple example like this provides the opportunity for a lot of design decisions, and in order to focus on the meat of the matter – editing and saving data back to a MySQL back end – I took the easy way out in terms of UI so as to not distract us. I didn’t include all of the tiny bits of polish that a production version of this form would have. I’ll mention a couple of these shortcuts as we develop the form, and we’ll add those in down the road with future examples. The first shortcut is that we’re going to load up the list box with all of the records in the business and location tables. There are only a few hundred records in this sample database, so doing a SELECT ALL won’t pose much of a burden on the form. This enables us to avoid the use of an initial form to filter the database first. This means we call this form just like the ui_add form: do form ui_edit with 'localhost', 'whil', 'secret'

passing parameters that are handled in the init() of the form. The corresponding data for the currently highlighted row in the list box displays in the controls to the right, just like in our navigation form in Chapter 18. In this form, however, the user can edit a value in one of the controls; once they tab out of the control, the Save button is enabled if they changed the data. As the user highlights businesses in the list box, the values in the controls on the right side are displayed for the selected business in the list box. The list box displays values from the denormalized cursor, csrRes, which contains data from a join of the BIZ and LOC tables. As you see in Figure 2, it is possible to see the same business name more than once, each instance representing a different location.


This denormalized architecture makes it easy for the user to find a business (and we’ll expand on this in future examples to allow searching by street name), but it makes the construction of the underlying code a bit more difficult. The user can edit either the Business Name/Second Line values from the BIZ table, or any of the attributes in the LOC table, and not be aware that there is more than one table under the hood supporting the form. However, when the user saves a record, we have to figure out which (if not both) tables need to be updated. But we’re database jocks, so I suspect we can handle this little chore. The form also has a Done button that simply closes the form (after attending to other chores, such as closing the connection and cleaning up object references). For the time being, uncommitted changes are just abandoned.

Internal design Under the hood, this form is based on the hwfrmnav class. If you’re building this yourself, create the form like so: create form ui_edit as hwfrmnav from hwctrl.vcx

Again, add the properties ih and ocslib, and include the same code for the Init() method as for the ui_add form. Then make one simple change to the Init() code. In the ELSE clause of testing for connection success, call the form’s custom getresults() method in order to populate the list box and fill the controls with the data from the result cursor’s first record. The code snippet looks like this: else thisform.iH = m.liH if !thisform.getresults() return .f. endif endif

If the SQLEXEC() function call in the getresults() method fails, the user is notified and the form isn’t opened. Before we get to explaining the getresults() method in detail, let’s finish up with the layout of the form. As you’ve seen, when you create the edit form from the hwfrmnav class, the list box and Done button come along for the ride, as do a pair of developer-only primary key fields. Place the various controls on the form as shown in Figure 2. The LostFocus() method of each data entry control calls the custom validate_data() method. This routine detects differences between the values in the controls and the underlying database records and enables the Save button if there are any differences. This is the second quick and dirty decision – the controls don’t do any additional validation, such as requiring a zip code to be of a certain format or ensuring that the street direction is limited to the values enumerated in the MySQL database (E/N/S/W or empty.)

Displaying data on the form Back to getresults(). It JOINs the BIZ and LOC tables, creating a denormalized cursor (csrRes) for every Location record in the database, duplicating the business information as necessary. The primary keys for the Business and Location records are also placed in this


cursor. Then the array that supports the results list box (hwlstnavres.aItems) is populated, and the highlight is placed on the first row in the list box. * ui_edit.getresults() debugout this.Name+'.getresults' text to m.lcStr noshow select biz.iidbiz, biz.cnabiz, biz.cnasec, loc.iidloc, loc.cno, loc.edir, loc.cstreet, loc.csuf, loc.csecline, loc.ccity, loc.cstate, loc.czip, iishq, tclosed from BIZ, LOC where BIZ.iidbiz = LOC.iidbiz order by biz.cNaBiz, loc.cStreet endtext m.liX = sqlexec(thisform.iH, m.lcStr, "csrRes") if m.liX > 0 * display if used("csrRes") and reccount("csrRes") > 0 m.liNumRows = reccount("csrRes") * fill array populating the listbox dimension thisform.hwlstnavRes.aItems[m.liNumRows,2] m.li = 1 scan thisform.hwlstnavRes.aItems[m.li,1] = csrRes.cNaBiz thisform.hwlstnavRes.aItems[m.li,2] = csrRes.iidBiz m.li = m.li+1 endscan else * display nothing dimension thisform.hwlstnavRes.aItems[1] thisform.hwlstnavRes.aItems[1] = [No Results.] endif else local aOops[1] m.liHowManyErrors = aerror(aOops) for m.li = 1 to m.liHowManyErrors insert into ZOOPS ; (inoerr, ctext, ctextodbc, csqlstate, inoerrsql, ihandle, tadded) ; values ; (aOops[m.li,1], aOops[m.li,2], aOops[m.li,3], aOops[m.li,4], ; aOops[m.li,5], aOops[m.li,6], datetime()) next * display nothing messagebox("SQL Execution failed; code is" +chr(10) +chr(13) +chr(10) +chr(13) ; +transform(m.lcStr)) return .f. endif thisform.hwlstnavRes.ListIndex = 1

Highlighting the first row in the list box causes the controls to be populated with data. How? The list box is based on the hwlstnav class, which contains the anychange() method. anychange() presumes the existence of a form-level method, tabletoform(). Highlighting the first row calls the anychange() method, which in turn calls tabletoform(). And tabletoform() moves data from the array into the controls on the right side of the form. tabletoform() also populates the developer text boxes for the primary keys for the BIZ and LOC tables.


* ui_edit.tabletoform() lparameters m.tiCurRow debugout this.Name+'.tabletoform' * if tiCurRow = 0, the cursor is empty * the array has been filled * we are redisplaying the form with a new row highlighted * and new values in the individual controls that we get from the local cursor * we may have to go back to the database for minor entity data thisform.hwtxtdevIID1.Value = csrRes.iidBiz thisform.hwtxtdevIID2.Value = csrRes.iidLoc thisform.hwtxtBusiness.Value = csrRes.cNaBiz thisform.hwtxtNaSec.Value = csrRes.cNaSec thisform.hwtxtNo.Value = csrRes.cNo thisform.hwtxtDir.Value = csrRes.eDir thisform.hwtxtStreet.Value = csrRes.cStreet thisform.hwtxtSuf.Value = csrRes.cSuf thisform.hwtxtSecLine.Value = csrRes.cSecLine thisform.hwtxtCity.Value = csrRes.cCity thisform.hwtxtState.Value = csrRes.cState thisform.hwtxtZip.Value = csrRes.cZip thisform.hwchkHeadquarters.Value = csrRes.iishq thisform.hwtxtClosed.Value = ttod(csrRes.tClosed)

Saving data Obviously the Save button calls the save method, but before we get to the action in Save(), note that the anychange() method of the list box also calls Save() if the Save button is enabled. This is the third quick and dirty decision – if the user makes a change to a value in one record and then navigates to another record in the list box without explicitly saving the record, we just go ahead and save the data for them. Some folks would likely want to ask the user if they want to save before allowing them to move to another record, but that extra work kind of gets in our way for the time being. The custom Save() is where the action is today. First, good values will be created from the data the user has entered. For example, each character string is passed through the z_es() function to escape quotation marks, so the user can save a value like Bob's Extraordinary Pizza Parlor

Now the good stuff happens. Once we have valid data to stuff back into the MySQL database, we use the SQL UPDATE command to do the actual stuffing. It’s time to ask ourselves, “Self, what record should we be updating?” In this case, we have to ask ourselves this twice, because we’re updating both the BIZ and the LOC tables. Fortunately, we know what records in the back-end tables we’re interested in because the primary keys for the records in question are in the developer-only text boxes. (When this application runs live, the text boxes are still available for the application to access – they’re just not visible or enabled.) We create an UPDATE command for the BIZ table, store it to m.lcStr, and execute it via SQLEXEC(). If the commit works, we store .t. to the m.llSaveWentWell variable, and .f. otherwise. Then we do the same UPDATE, m.lcStr, and SQLEXEC() things for the LOC table. This time, however, if the command executes successfully, we just leave the value of m.llSaveWentWell as is, since it’s still true. If the second commit failed, store a .f. to the variable so we know we had at least one failure. Finally, alert the user if m.llSaveWentWell


was set to .f. anywhere along the line and disable the Save button. (This all should be wrapped in a transaction, but one step at a time. We’ll deal with transactions in the next chapter.) Finally, the error reporting mechanism shown here could be substantially more robust, of course. An exercise left to the reader, I suspect. * ui_edit.save() debugout this.Name + '.save' m.liidBiz = thisform.hwtxtdeviid1.value m.liidLoc = thisform.hwtxtdeviid2.value m.lcNaBiz = thisform.ocslib.z_es(alltrim(thisform.hwtxtBusiness.Value)) m.lcNaSec = thisform.ocslib.z_es(alltrim(thisform.hwtxtNaSec.Value)) m.lcNo = thisform.ocslib.z_es(alltrim(thisform.hwtxtNo.Value)) m.leDir = thisform.ocslib.z_es(alltrim(thisform.hwtxtDir.Value)) m.lcStreet = thisform.ocslib.z_es(alltrim(thisform.hwtxtStreet.Value)) m.lcSuf = thisform.ocslib.z_es(alltrim(thisform.hwtxtSuf.Value)) m.lcSecLine = thisform.ocslib.z_es(alltrim(thisform.hwtxtSecLine.Value)) m.lcCity = thisform.ocslib.z_es(alltrim(thisform.hwtxtCity.Value)) m.lcState = thisform.ocslib.z_es(alltrim(thisform.hwtxtState.Value)) m.lcZip = thisform.ocslib.z_es(alltrim(thisform.hwtxtZip.Value)) m.liisHQ = thisform.hwchkHeadquarters.Value m.lcClosed = thisform.ocslib.z_tfed(thisform.hwtxtClosed.Value) text to m.lcStr textmerge noshow update BIZ set cNaBiz = '<>', cNaSec = '<>', cchanged = 'bob', tchanged = now() where biz.iidbiz = <> endtext m.liX = sqlexec(thisform.iH, m.lcStr) if m.liX > 0 * continue on m.llSaveWentWell = .t. else local aOops[1] m.liHowManyErrors=AERROR(aOops) =thisform.ocslib.z_sqlerror(@aOops, m.lcStr, m.liHowManyErrors) m.llSaveWentWell = .f. endif text to m.lcStr textmerge noshow update LOC set cNo = '<>', eDir = '<>', cStreet = '<>', cSuf = '<>', cSecLine = '<>', cCity = '<>', cState = '<>', cZip = '<>', iishq = <>, tClosed = '<>', cchanged = 'bob', tchanged = now() where loc.iidloc = <> endtext


m.liX = sqlexec(thisform.iH, m.lcStr) if m.liX > 0 * continue on * don't need to update savewentwell * if previous save was good, so was this, so still .t. * if previous save failed, we're going to return a .f. * regardless if this one worked else local aOops[1] m.liHowManyErrors=AERROR(aOops) =thisform.ocslib.z_sqlerror(@aOops, m.lcStr, m.liHowManyErrors) m.llSaveWentWell = .f. endif if m.llSaveWentWell thisform.hwlblSaveFailed.caption = "" else thisform.hwlblSaveFailed.caption = "An error has happened during save. Please see the error log." endif thisform.hwcmdSave.Enabled = .f. return m.llSaveWentWell

Once the data is saved, you may notice that, in the event of a business name change, the list box isn’t updated afterwards. Again, you might want to try your hand at making this happen. If not, never fear; we add this in shortly, in a future example. As before, the Close() method, called from the Done button, closes the cursor and disconnects from the database. And that’s all there is to our first editing form. An alternative to using the “developer-only” text boxes is setting up properties in the form to represent the primary key values; I like the text boxes because it enables easier debugging for those times when a typo results in the wrong primary key being stuffed into a table. It’s now time for you to open up the source for this example and play around with it, adding a list box refresh, perhaps, or changing the Street Direction text box to a combo box that limits the choices available to the user to those allowed by the database’s field definition. Once you’re comfortable, you might consider changing the interface to include separate buttons for saving the business data versus the location data, or maybe even reworking the interface so it’s not denormalized as it is here.

A simple two table delete form (ui_delete.scx) By now you’re getting the hang of it. Once you make a connection and haul the data into your form, you can work with it via a local cursor just as you would local tables. When you’re ready to commit your changes, you fire back to the database and you’re done. Our delete form is just like the ui_edit form, except with a new Delete button and the corresponding Delete() method. We’ll also add the call to refresh the list box afterwards because it wouldn’t do to allow the user to try to delete the same record twice. Figure 3 shows the finished ui_delete form.


Figure 3 . A simple two table delete form.

Denormalized table issues We simply (don’t you get a chill down your spine when you hear ‘simply’?) call the custom Delete() method when the user hits the Delete button. Inside that method, though, there are some decisions to be made based on this parent-child interface. Since the list box is denormalized, the list can contain two types of rows. The first type of row contains a name of a business that has only one location; the other type of row contains a name of a business that has multiple locations (although the row in question represents just one of those locations.) If the highlighted row in the list is the first type, the work to perform when the user clicks the Delete button is easy – delete both the location and the business it belongs to. The second, though, is trickier. Do we want to delete the location but leave the business record intact, since there are other children (locations) attached to the business? Probably. After all, we should be aware that the reason the user is deleting a record is most likely because the location closed, fell into a sinkhole, or was bought by someone else. (The user might also be deleting the record because it was entered in error.) The bottom line is that this specific location is no longer affiliated with the business. What if, however, the entire business was shut down? In this scenario, it would be convenient to give the user the option to delete all locations for this business, and then delete the business itself. It may be the case that you decide on this rule as an absolute for the entire system, and structure the database that way – business has a one-to-one or -more relationship to location. MySQL, like VFP, has the ability to set up referential integrity on the database server, so all of the locations (children) are automatically deleted when the business (parent) is deleted, or that the parent business is deleted when no child locations are left. In this example, we’ll do it manually.


First check if there is only one location, and if so, delete the biz and the location. If there is more than one location, we just delete the location and leave the business alone. This form is an enhancement of the ui_edit form, which means that we call this form the same way: do form ui_delete with 'localhost', 'whil', 'secret'

passing the parameters that are handled in the Init() of the form.

The Delete() method Under the hood, this form is virtually identical to the ui_edit.scx form, so if you’re building this yourself, create the form by making a copy of ui_edit.scx, calling the result ui_delete.scx. The only differences between this and the ui_edit form are the Delete button and the Delete() method. The Delete button’s Click() method calls the form’s Delete() method. The Delete() method grabs the primary keys for the BIZ and LOC tables, and then stores the current ListIndex value of the list box, so we can return the highlight to a logical place after deleting the current row. Then the ListIndex value is tested to make sure a row in the list box is highlighted, so we know which row we want to delete. As soon as the user clicks the Delete button, we ask the user to confirm their intention for deletion, count how many locations belong to this record’s Business, and then delete the location. Then, if there’s just this one location for the business, delete the business too. We set a flag upon successful deletion of the location so we don’t accidentally delete a business after being unable to delete the child location. We also don’t delete the business if this location has sibling locations. Finally, we call the form’s custom getresults() method to repopulate the list box, and set the highlight to the same row in the list box, which means the row after the row being deleted becomes the new highlighted row. (In order to keep things simple, this example doesn’t handle the case where you were already on the last row of the list, and then deleted the last row. In such a situation, you would want to check to see if this.ListIndex = this.ListCount, and if so, set the ListIndex = m.liOriginalListIndex - 1.) * ui_delete.delete() debugout thisform.Name + '.delete' m.liidBiz = thisform.hwtxtdeviid1.value m.liidLoc = thisform.hwtxtdeviid2.value m.liOriginalListIndex = thisform.hwlstnavres.ListIndex * * * * * * *

make sure on a record determine if > 1 loc get name of biz/loc ask user to confirm delete refresh list box reposition to previous record

if thisform.hwlstnavres.ListIndex = 0 return endif m.lcToDelete = thisform.hwtxtBusiness


if messagebox("Are you sure you want to delete " + chr(13) + chr(10) ; + thisform.hwtxtBusiness.Value + chr(13) + chr(10) ; + " on " + thisform.hwtxtStreet.Value + thisform.hwtxtSuf.Value + chr(13) + chr(10) ; + "forever?",4+256) <> 6 && not yes (no) return .f. endif text to m.lcStr textmerge noshow select count(iidloc) as cHowMany from LOC where loc.iidbiz = <> endtext m.liX = sqlexec(thisform.iH, m.lcStr, "csrCount") if m.liX > 0 * continue on if val(csrCount.cHowMany) > 1 m.llThereIsJustOne = .f. else m.llThereIsJustOne = .t. endif else local aOops[1] m.liHowManyErrors=AERROR(aOops) =thisform.ocslib.z_sqlerror(@aOops, m.lcStr, m.liHowManyErrors) * we don't know how many others there are, so assume there is more than one m.llThereIsJustOne = .f. endif * delete the loc regardless m.llDeleteLocWasGood = .f. text to m.lcStr textmerge noshow delete from LOC where loc.iidloc = <> endtext m.liX = sqlexec(thisform.iH, m.lcStr) if m.liX > 0 * continue on m.llDeleteLocWasGood = .t. else local aOops[1] m.liHowManyErrors=AERROR(aOops) =thisform.ocslib.z_sqlerror(@aOops, m.lcStr, m.liHowManyErrors) * couldn't delete the LOC m.llDeleteLocWasGood = .f. endif m.llDeleteBizWasGood = .f. if m.llThereIsJustOne and m.llDeleteLocWasGood debugout "there is just one and del loc was good so deleting biz" * delete the biz too text to m.lcStr textmerge noshow delete from BIZ where biz.iidbiz = <> endtext m.liX = sqlexec(thisform.iH, m.lcStr) if m.liX > 0 * continue on m.llDeleteBizWasGood = .t. else local aOops[1] m.liHowManyErrors=AERROR(aOops) =thisform.ocslib.z_sqlerror(@aOops, m.lcStr, m.liHowManyErrors)


* couldn't delete the LOC m.llDeleteBizWasGood = .f. endif else * don't delete the biz cuz * - there are other, or * - the Loc delete wasn't successful endif * refresh list box thisform.getresults() * reposition if m.llDeleteLocWasGood thisform.hwlstnavres.ListIndex = m.liOriginalListIndex thisform.hwlstnavRes.SetFocus() if used("csrCount") use in csrCount endif

There are two schools of thought with respect to asking the user to confirm a deletion. The first is not to ask for confirmation; don’t insult the user’s intelligence – they clicked the button, assume they know what they’re doing. The other school, though, is that they may have inadvertently clicked the Delete button when they meant to click somewhere else nearby. With MySQL, not allowing the user the recover from such a mistake via a confirmation dialog is doubly important, because there is no ‘undo’ functionality in MySQL. Specifically, a delete in MySQL (as with other SQL databases) is not like a ‘delete’ in VFP. ‘Delete’ in MySQL is like ‘delete’ followed by ‘pack’ in VFP. There is no ‘delete flag’ in the MySQL database architecture. As a result, it’s critical that you make sure to have the user confirm the pending deletion with an “Are you sure?” dialog. Some folks find it handy to use an alternative method, simulating VFP’s delete flag, by using a “iIsDeleted” flag in each table, setting that flag to 1 to indicate the record is deleted, and when doing SQL SELECTS, making sure to include the “where iIsDeleted = 0” clause all the time. This will only work, however, if you’re not using MySQL’s Referential Integrity mechanisms. What else might you want to do in your spare time? Alerting the user that there are multiple locations of the business and asking them if they want to delete all of the locations is one possibility. Or you could notify the user if they are deleting the HQ location, so they can assign another location as the HQ if they so wish. You might also try allowing the user to click on multiple records in the list box, and then delete all that are marked with a single click of the delete button. Again, you want to make sure you require the user to confirm their intent. In a business sense, note that deletion may not be what some users want to do; rather, they’ll prefer to mark a location ‘closed’ by entering a Closed date. But we needed to learn how to delete records, right?


An all-in-one form (ui_allinone.scx) Each of the previous forms works pretty well, but real life is rarely that simple. Now it’s time to put add, edit, and delete capabilities into one form, together with a database schema that includes parents, children, and grandchildren.

Design of an all-in-one form As anyone who has attempted to put together a ‘name and address’ database knows, there are so many combinations and permutations of the designs that it’s nearly impossible to do justice to every scenario. As a result, you’ll have to live with the tacit assumptions I’ve made in designing the database and assembling the interface for this particular application. This “all-in-one” form starts out life as the ui_delete form we just looked at, but has a number of extensions added to it. First, we don’t dump the entire BIZ/LOC join cursor into the list box; instead, we allow the user to filter the list box by the first letter. Within that first letter, we display all business/location combinations like in previous examples, so it’s possible to see several “Starbucks” rows in the list box, albeit each row representing a different location. On second thought, given the proliferation of coffee shops, seeing several Starbucks is pretty much inevitable. We also allow the user to choose if they want the first letter to filter the business name, or the street name, so the user can view all of the businesses that begin with ‘Q’, or all of the businesses located on streets that begin with the letter ‘S’. In the former case, the list is sorted by business name, and then by street under each business. In the latter case, the list is sorted by street name, then by direction, and then by street number, so that 76 East Silver Spring Drive will show up before 107 West Silver Spring Drive, and both of those will appear before 43 North Snowshoe Circle. While this interface wouldn’t do for databases with tens or hundreds of thousands of records, the small sample tables included here will work fine. This form brings back the BizType child and Coordinate and Person grandchild entities seen in Chapter 18. The Business Type list box, populated from the BIZCAT table (which does a lookup into the ZLOOKUP table), contains one or more records that identify the type of business (restaurant, grocery, physicians office, etc.) The tables that support the Contacts and Folks list boxes are both related to the LOC table. “Contacts” are what some people call ‘coordinates’ – telephone numbers, email addresses, Web URLs, and so on. Each location likely has its own voice and fax numbers, and may well have its own Web site. Others, however, may have a single Web site for all locations. It is possible to create a design where some coordinates are attached to the Biz record (like the Web address or a main phone number) while other coordinates stay with a specific location (such as the fax machine at a specific store). However, this gets more complicated than what I want to deal with in this example. As a result, those coordinates that normally would be attached directly to the business are attached to the location identified as the headquarters for the business. “Folks” are the people who belong to the business; again, they most likely would be attached to a specific location, although some folks who roam between multiple locations might be attached to the headquarters location for practicality.


You’ll find “add (+)” and “delete (–)” buttons to the right of each set of controls for a table. The Add buttons open a new dialog that allows the user to add one or more records to that entity; to wit, the Add button to the right of the Business text boxes opens the Add dialog that we started out with in this chapter, while the Add button to the right of the Folks list box allows the user to add one or more people to the current location, using the same “Save and Close” and “Save and Add Another” mechanisms. Edits to the Business and Location controls in the list box enable and are saved by the “Save” button at the bottom of the form; changes to the other controls are made only by adding and deleting entire records. The entire form is shown in Figure 4.

Figure 4 . The "all-in-one" form provides add/edit/delete capabilities for multiple related tables. A number of niceties have been added to this form. For example, each of the letters at the top of the Business/Location list box act as a filter control, much like some Web sites use. After clicking a letter at the top of the Business/Location list box, that letter is disabled. Figure 4 shows the letter ‘R’ disabled after clicking it displays all of the businesses that begin with the letter ‘R’. The street addresses of businesses are displayed in the list box to allow the user to distinguish between multiple locations of the same business. I considered using two list boxes; one solely for business names and a second to display locations, but decided against it after seeing the capability to filter and sort the list by street name was more important than the ‘purist’ approach of normalizing the business name and street address combination.


If a business doesn’t have a location attached, no street address displays in the left-hand list box. “Ralph’s Fine Automobiles” and “Royal Palace” are two such examples. You’ll see that the street numbers are right justified, making it easier to distinguish between, say, “57 West Fifth” and “511 West Fifth.” It also aids in sorting the list; the user can see the street numbers progress from smallest to largest. See Figure 5.

Figure 5 . Street numbers are right justified. List boxes display the text “No Results” if there are no records for that entity.

Internal design Let’s take a look under the hood. This form is yet another enhancement to the ui_edit form, so it’s based on the hwfrmnav class in hwctrl.vcx. This means it’s called like so: do form ui_allinone with 'localhost', 'whil', 'secret'

The form has one new property, cWhichLetter, that identifies which filter letter is currently active. There are a number of new methods, all dealing with the deletion of records; I’ll cover those shortly. Much of the same code from ui_edit is still found in ui_allinone, so I’ll just point out the differences. In the Init(), the new form property, cWhichLetter, is assigned its initial value, ‘A’, and the ‘A’ label is initially disabled. * ui_allinone.init() (partial) if m.liH < 1 messagebox("No connection; handle is:"+transform(m.liH)+":") return .f. else thisform.iH = m.liH


thisform.Caption = alltrim(thisform.Caption) + " (Handle: " + transform(thisform.iH) + ")" thisform.hwlblSaveFailed.Caption = '' thisform.cWhichLetter = 'A' thisform.hwlblpickA.ForeColor = RGB(0,0,0) thisform.hwlblpickA.FontUnderline = .f. if !thisform.getresults() return .f. endif endif

The getresults() method has been modified to handle the letter and the ‘Business or Street’ filters. The letter filter is assigned to a variable, like so: if thisform.hwopgCompanyOrStreet.Value = "Company Name" m.lcFilter = "cNaBiz like '" + thisform.cWhichLetter + "%'" else m.lcFilter = "cStreet like '" + thisform.cWhichLetter + "%'"

Choosing which WHERE clause to use is based on the value of the Business or Street option group. * ui_allinone.getresults() (partial) text to m.lcStr textmerge noshow select biz.iidbiz, biz.cnabiz, biz.cnasec, loc.iidloc, loc.cno, loc.edir, loc.cstreet, loc.csuf, loc.csecline, loc.ccity, loc.cstate, loc.czip, iishq, tclosed from BIZ left join LOC on (LOC.iidbiz = BIZ.iidbiz) where <> order by biz.cNaBiz endtext

The tabletoform() method that moves data from the csrRes cursor created by getresults() has been modified to also query the BIZCAT, COOR, and PERSON tables for the records related to the chosen Business and Location. * ui_allinone.tabletoform() (partial - just fill one listbox) * coordinates listobx text to m.lcStr noshow select iidcoor, cdata, etype from COOR where coor.iidloc = ?thisform.hwtxtdevIID2.value endtext m.liX = sqlexec(thisform.iH, m.lcStr, "csrResCoor") if m.liX > 0 * display if used("csrResCoor") and reccount("csrResCoor") > 0 m.liNumRows = reccount("csrResCoor") * fill array populating the listbox dimension thisform.hwlstCoor.aItems[m.liNumRows,2] m.li = 1 scan thisform.hwlstCoor.aItems[m.li,1] = csrResCoor.eType + " " + csrResCoor.cData thisform.hwlstCoor.aItems[m.li,2] = csrResCoor.iidCoor


m.li = m.li+1 endscan else * display nothing dimension thisform.hwlstCoor.aItems[1] thisform.hwlstCoor.aItems[1] = [No Results.] endif else local aOops[1] m.liHowManyErrors = aerror(aOops) for m.li = 1 to m.liHowManyErrors insert into ZOOPS ; (inoerr, ctext, ctextodbc, csqlstate, inoerrsql, ihandle, tadded) ; values ; (aOops[m.li,1], aOops[m.li,2], aOops[m.li,3], aOops[m.li,4], ; aOops[m.li,5], aOops[m.li,6], datetime()) next * display nothing dimension thisform.hwlstCoor.aItems[1,2] thisform.hwlstCoor.aItems[1,1] = [Bad Results.] thisform.hwlstCoor.aItems[1,2] = 0 endif thisform.hwlstCoor.Requery() thisform.hwlstCoor.ListIndex = 1

Adding and deleting minor entities The Add buttons each contain a couple lines of code in the Click() method. This code calls a separate Add form for that entity. However, the Delete buttons each call a separate delete method in their Click() method. Why the difference? Well, because I’m lazy. Strictly speaking, I should have created separate add methods that were called from the Add button clicks. Code that does ‘work’ doesn’t belong in the Click() of a command button. The function of a command button, as Andy Kramek so eloquently puts it, is to notify some object that an action is needed. If the add code got any longer or more complex, I’d move it into separate methods where it belongs.

Adding a business Clicking the Add Business button brings forth the same form shown in Figure 1. However, the actual code that calls the form is slightly different. This is in the Click() method of the button: * ui_allinone.hwcmdaddbiz.click() do form ui_allinone_addbiz with thisform.iH, thisform.hwtxtdeviid1.value, thisform.ocslib to m.liidnew if m.liidnew > 0 thisform.getresults() else * no biz added endif

As you can see, the Add Business form is called with parameters that reference the calling form’s connection handle and the ocslib object. (The reference to the primary key isn’t necessary in this specific form but will be used in other “Add” forms. I kept it in there in case I wanted to abstract the whole function.) Doing so enables the Add Business form to


communicate with the database server using the same connection, and have access to the client-server functions created in the Init() of the calling form. Inside the Add Business form, everything functions pretty much like it did in the standalone example earlier in this chapter, with one important exception. The Save() method, after successfully inserting the new business record, does a second call to the database, requesting the primary key of the record just added, like so: * ui_allinone_addbiz.save() if m.liX > 0 * insert was successful text to m.lcStr noshow select last_insert_id() as cLastID endtext m.liX = sqlexec(thisform.iH, m.lcStr, "csrLastID") if m.liX > 0 thisform.iLastID = val(csrLastID.cLastID) else thisform.iLastID = 0 endif

This value, iLastID, is returned by the Add Business form to the calling form (in the Unload method). The calling form, ui_allinone, now knows the last record added to the BIZ table, and this value can be used to navigate to that record and reposition the interface on that record.

Deleting a business The Delete Business button calls the deletebiz() method, which contains pretty much the same code as in the ui_delete form. The only difference is that instead of just deleting the location attached to the business (if there’s only one location) the method also deletes all of the child records – Business Types attached to the Business as well as Contacts and Folks attached to the single location. * ui_allinone.deletebiz() debugout thisform.name + '.deletebiz' m.liidBiz = thisform.hwtxtdeviid1.value m.liidLoc = thisform.hwtxtdeviid2.value m.liOriginalListIndex = thisform.hwlstnavres.ListIndex * make sure on a record if thisform.hwlstnavres.ListIndex = 0 return endif * force confirm if messagebox("Are you sure you want to delete the business " + chr(13) + chr(10) ; + thisform.hwtxtBusiness.Value + chr(13) + chr(10) ; + "forever?",4+256) <> 6 && not yes (no) return .f. endif * determine how many loc m.llThereIsAtMostOne = .t.


text to m.lcStr textmerge noshow select count(iidloc) as cHowMany from LOC where loc.iidbiz = <> endtext m.liX = sqlexec(thisform.iH, m.lcStr, "csrCount") if m.liX > 0 * continue on m.liHowManyLoc = val(csrCount.cHowMany) * note that this ID value is valid only if there was one loc for the biz else local aOops[1] m.liHowManyErrors=AERROR(aOops) =thisform.ocslib.z_sqlerror(@aOops, m.lcStr, m.liHowManyErrors) * we don't know how many others there are, so assume there is more than one m.liHowManyLoc = -1 m.liidLoc = 0 endif * delete * will delete if 0 or 1 locations * will warn and not allow if > 1 location do case case m.liHowManyLoc < 0 messagebox("Unable to delete business because count of locations failed.") return .f. case m.liHowManyLoc = 0 * no loc to delete, just going to delete the biz * delete the biz types thisform.deletebizcat() case m.liHowManyLoc = 1 * going to delete the coordinates thisform.deletecoor() * delete the persons thisform.deleteperson() * delete the sole loc thisform.deleteloc() * delete the biz types thisform.deletebizcat() otherwise * lots of locations, so we don't delete locations * AND we don't delete business messagebox("There is more than one location attached to this business. Deleting the business would leave the rest of the locations without a business to belong to. Delete cancelled. Delete the other locations first, if you truly want to delete the business.") return .f. endcase * now delete the biz m.llDeleteBizWasGood = .f. text to m.lcStr textmerge noshow delete from BIZ where biz.iidbiz = <> endtext m.liX = sqlexec(thisform.iH, m.lcStr) if m.liX > 0 * continue on


m.llDeleteBizWasGood = .t. else local aOops[1] m.liHowManyErrors=AERROR(aOops) =thisform.ocslib.z_sqlerror(@aOops, m.lcStr, m.liHowManyErrors) * couldn't delete the LOC m.llDeleteBizWasGood = .f. endif * refresh list box thisform.getresults() * reposition thisform.hwlstnavres.ListIndex = m.liOriginalListIndex thisform.hwlstnavRes.SetFocus() if used("csrCount") use in csrCount endif

Adding and deleting business types The Add Business Type form, called from the Add button next to the Type of Biz field, is shown in Figure 6.

Figure 6 . The Add Business Type form allows the user to add one or more Business Types to a Business. To understand what’s happening in this form, it’s important to revisit the structure of the BIZCAT table. The fields of interest with a couple of sample records are displayed here: iidbizcat 1

iidbiz 10

iidcat 24


2

10

57

The first field is a surrogate primary key for the BIZCAT table. The second field points to the primary key for the BIZ table, tying this Business Type to a specific Business record. The third field points to the primary key for the ZLOOKUP table – identifying the Business Type record in the lookup table. Thus, for iidbizcat = 1, iidbiz = 10, pointing to record 10 in BIZ (such as “Benji’s Deli”), while iidzlookup points to record 24 (say, “Bakery”). The second record is also for Benji’s Deli, and the iidzlookup record, # 57, points to “Restaurant.” Unlike the Add Business form, the “hwtxtdeviid1.value” parameter passed to this form is important, because the value holds the primary key of the current business record. As a result, it becomes one of the foreign keys in the BIZCAT table for the new BIZCAT record. And what about the other foreign key? Hold your horses for just a moment. The Add Business Type form also differs from the Add Business form in that it goes out to the database and queries the ZLOOKUP table for all valid CAT (Business Type) records in order to populate the list box. The default value in the list box, NEC, stands for “Not Elsewhere Classified” and serves as a default for Businesses whose owners don’t know what business they’re in yet. (Eric Selje correctly notes that, outside of Silicon Valley, this is considered to be a bad business plan.) * ui_allinone_addbiztype.init() lparameters m.tiH, m.tiIDBiz, m.toCS debugout this.Name + '.Init' thisform.iH = m.tiH thisform.ocslib = m.toCS thisform.iidparent = m.tiidbiz dodefault() thisform.Caption = alltrim(thisform.Caption) + " (Handle: " + transform(thisform.iH) + ")" thisform.hwlblSaveFailed.Caption = '' * populate the Business Type cbo m.liHowMany = 0 text to m.lcStr textmerge noshow select cde, iidzlookup from ins.zlookup where cnalookup = 'CAT' order by cde endtext m.liX = sqlexec(thisform.iH, m.lcStr, "csrResCat") if m.liX > 0 * display if used("csrResCat") and reccount("csrResCat") > 0 m.liNumRows = reccount("csrResCat") * fill array populating the listbox dimension thisform.hwlstBizType.aItems[m.liNumRows,2] select csrResCat m.li = 1 scan thisform.hwlstBizType.aItems[m.li,1] = csrResCat.cDe thisform.hwlstBizType.aItems[m.li,2] = csrResCat.iidzlookup m.li = m.li+1 endscan else


dimension thisform.hwlstBizType.aItems[1,2] thisform.hwlstBizType.aItems[1,1] = "No Results." thisform.hwlstBizType.aItems[1,2] = 0 endif else local aOops[1] m.liHowManyErrors=AERROR(aOops) =thisform.ocslib.z_sqlerror(@aOops, m.lcStr, m.liHowManyErrors) dimension thisform.hwcboBizType.aItems[1,2] thisform.hwcboBizType.aItems[1,1] = "No Results." thisform.hwcboBizType.aItems[1,2] = 0 return .f. endif thisform.hwlstBizType.Requery() thisform.hwlstBizType.ListIndex = 1 m.liRowNumber=ascan(thisform.hwlstBizType.aItems, "NEC", -1, -1, 1, 8) thisform.hwlstBizType.ListIndex = m.liRowNumber thisform.hwlstBizType.Refresh()

The second column in the array that supports the Business Type list box is populated with the primary key of the ZLOOKUP table for the appropriate Business Type records. This key becomes the foreign key pointing to the ZLOOKUP table – the second foreign key in the BIZCAT table. As a result, the Save() method ends up just stuffing a couple of foreign keys into the BIZCAT table. * ui_allinone_addbiztype.save() (partial) * save data debugout this.Name + '.save' m.liidbiz = thisform.iidparent m.liidcat = thisform.hwlstBizType.aitems[thisform.hwlstBizType.listindex,2] text to m.lcStr textmerge noshow insert into INS.bizcat (iidbiz, iidcat, cadded, tadded, cchanged, tchanged) values (<>, <>, 'bob', now(), 'bob', now() ) endtext m.llSaveWentWell = .f. m.liX = sqlexec(thisform.iH, m.lcStr) if m.liX > 0 * continue on text to m.lcStr noshow select last_insert_id() as cLastID endtext m.liX = sqlexec(thisform.iH, m.lcStr, "csrLastID") if m.liX > 0 thisform.ilastid = val(csrLastID.cLastID) else thisform.ilastid = 0 endif return .t. else local aOops[1] m.liHowManyErrors=AERROR(aOops) =thisform.ocslib.z_sqlerror(@aOops, m.lcStr, m.liHowManyErrors) return .f.


endif

Adding and deleting contacts The Add Contact form is shown in Figure 7.

Figure 7 . The Add Contact form lets the user add one or more coordinates, such as telephone numbers or email addresses, to a Location. Analogous to the Add Business Type form, which is passed the hwtxtdeviid1.value parameter, the hwtxtdeviid2.value parameter passed to this form is used as the foreign key in the COOR table when a new contact record is added, because COOR is tied to LOC (whose primary key is found in iid2, compared to iid1 that contains the primary key for BIZ). The Contact Type combo box is also populated via a lookup against the ZLOOKUP table, in order to provide friendly labels for the various contact types. Unlike the Add Business Type form, though, the foreign key from the ZLOOKUP table for the Contact Type isn’t stuffed into the new contact record; instead, the actual text value is inserted. The text for various Business Types is likely to change for at least some Business Types, so we put the primary key to the Business Type record in the BIZCAT table. On the other hand, the strings “voice” and “fax” aren’t likely to change any time in the foreseeable future, and putting the actual data in the COOR table makes reporting and lookups just that much easier. Using a combo box instead of a text box where the user could enter free form data ensures consistency in the values of Contact Type. Using a lookup table to populate the combo box ensures that we can add new contact types easily. Thus, the save method for contact stuffs the FK to COOR and the text for Contact Type and Contact data into a new record in COOR. The Add Location (Figure 8) and Add Person (Figure 9) forms function identically to the Add Business form, providing controls to enter the appropriate data.


Figure 8 . The Add Location dialog lets the user add a physical location to an existing business.

Figure 9. The Add Person dialog lets the user add a contact person to a specific location.


Tricks and Tips The examples in this chapter have incorporated a number of tricks that were sort of blown by in our haste to build the forms and demonstrate the proper client-server architecture. It’s worth a second look at these.

Storing a street number field so it sorts properly Handling addresses isn’t a straightforward endeavor if you want to be able to sort on the street numbers. Typically, you want to see a list of addresses on a specific street in the order you’d see them if you were walking or driving along the street. Thus, within a specific street name, you want all of the addresses in each direction together, so you can see the addresses on “East Main” together, and then the addresses on “West Main” next in the list. First off, you need to place the street number, direction, and street name in separate fields. This can already be more work than some folks are used to, or interested in performing. Within a direction and street, you want the street numbers sorted in numeric order. You have two choices. The first is to use a numeric field for the street number, and convert it to a character, appropriately padding the left with spaces when displaying. The other is the technique I used here – use a character field for the whole number and right justify the number when stored in the table, so spaces in front of small street numbers are sorted before larger street numbers, disregarding the size of the first digit. You’ll see the leading spaces before the ‘6’, ‘47’, and ‘88’ means those three addresses are sorted before the address with three digits in the number: 6 47 88 102

East East East East

Main Main Main Main

The only downside to character-typed street numbers is handling the data entry of the street number; you have to pad the character string with spaces on the left when storing it to the table.

Retrieving empty dates from MySQL As discussed in Chapter 12, MySQL stores empty dates differently than Visual FoxPro. Chapter 12 covered how to save VFP dates (including empty ones) in MySQL; what about the reverse – retrieving empty dates in a MySQL table and popping them into a VFP cursor? Suppose the MySQL field ‘tclosed’ contains an empty date. Looking at that field in the MySQL Query Browser, you’d see 0000-00-00. Running this SQL command against that table brings the field into the VFP cursor ‘csrDate’: =sqlexec(m.liH, "select tclosed from SOMETABLE", "csrDate")

If you look at the value in the cursor, you’ll see the following: ? csrDate.tClosed . . : :


which indicates the field is empty. However, using the VFP empty() function will make you say “Huh?” ? empty(csrDate.tClosed) .F.

Whoa. As Lao Tzu would say, if he had been a database programmer, “The Tao that appears empty but isn’t is not the true Tao.” When testing for an empty date in a VFP cursor that was created from a MySQL table, empty() is not going to do the trick. I’ve found testing against the largest legitimate date in VFP, 9999-12-31, works: ? iif(csrDate.tClosed > {^9999/12/31}, "Empty", "Not Empty") Empty

You’ll see this test in the enhanced version of the z_tfed (test for empty date) method in the cslib class library.

Retrieving null dates from MySQL Null dates are different from empty dates, so you need to treat them differently. Null dates can be created in MySQL in one of two ways. First, the MySQL date field could be defined as allowing nulls, and the record brought into a VFP cursor could contain a null date value. Or you could do a join between two tables where the resulting table has ‘missing’ records that contain date fields; those date fields will contain null values. Once a VFP cursor has a null value, all the same ol’ problems with nulls surface: inability to compare a null value with another value, special handling of null values in controls on data entry forms, frogs falling from the sky, those sorts of things. Fortunately, unlike empty(), the VFP function isnull() works fine on null fields brought into VFP cursors from MySQL. =sqlexec(m.liH, "select cnabiz, cstreet, tclosed from biz left join loc on loc.iidbiz = biz.iidbiz", "csrJoin")

Now, via a Browse window, navigate to a record in csrJoin that shows “.NULL.” in the field. ? isnull(csrJoin.tclosed) .T.

Voila! If you need to do more than simply test for a null, you can use the nvl() function: select nvl(datefield, {})

to do work with fields that may or may not contain nulls.

Handling full outer joins in MySQL Discovering the fact that MySQL does not handle full outer joins will break many a developer’s heart, particularly those used to being able to accomplish this in Visual FoxPro.


To refresh the memory of those of you who are a little rusty on these joins, suppose we have two tables, BIZ (business) and LOC (location). There should be one or more records in LOC for every BIZ record. However, now and again we may run across a BIZ record with no locations. A typical join, then, would only display the BIZ-LOC combinations which have at least one location for a business. Those businesses without at least one location are left out: select cnabiz, cstreet from BIZ join LOC on loc.iidbiz = biz.iidbiz

produces this type of listing: Acupuncture Unlimited Birds Of A Feather Can-Can Dance Studio

400 East Silver Spring 212 North Willow Grove 1801 Cannery Lane

A left outer join like this: select cnabiz, cstreet from BIZ left join LOC on loc.iidbiz = biz.iidbiz

will also display those businesses without a location: Acupuncture Unlimited Beyond The Ether Groceries Birds Of A Feather Can-Can Dance Studio

400 East Silver Spring 212 North Willow Grove 1801 Cannery Lane

What about the situation where a location is without a business? This is a legitimate scenario; just because a business moves out of a building doesn’t mean that building disappears. It’s now a location without a business. In some cases, it would be nice to see those locations as well. In VFP, this can be accomplished with a full outer join: select cnabiz, cstreet from BIZ full outer join LOC on loc.iidbiz = biz.iidbiz

which produces a listing like the following: Acupuncture Unlimited Beyond The Ether Groceries Birds Of A Feather

Can-Can Dance Studio

400 East Silver Spring 212 214 220 1801

North Willow Grove North Willow Grove North Willow Grove Cannery Lane

Unfortunately, MySQL doesn’t recognize the “full outer join” syntax. Instead, you need to do a union on a pair of left and right joins: select cnabiz, cstreet from BIZ left join LOC on loc.iidbiz = biz.iidbiz union select cnabiz, cstreet from BIZ right join LOC on loc.iidbiz = biz.iidbiz

to produce the same result as the VFP full outer join.


Selecting number of records in a result set It is a frequent need to know how many records are in a result set, or might be if you wanted to go through the work of doing the whole query. Issuing a simple SELECT that returns a single number as the answer is handy, because the work is done on the server and the network traffic that results is minimal. A typical SELECT would look like this: select count(iidbiz) as iHowMany from BIZ into cursor csrResCount

However, the result, csrResCount.iHowMany, is not an integer as you might have expected. Instead, it’s a character field, which will bite you when you try to compare the contents of the field to numeric values like, say, 0 or 100. To remind myself that the result is a character, I always name the result ‘cHowMany’, and then do the conversion upon the contents: =sqlexec(m.liH, "select count(iidbiz) as cHowMany from BIZ", "csrResCount") m.liHowMany = val(csrResCount.cHowMany)

Sometimes you want to run a query with a “limit” clause so only a few rows are returned in the result set. However, you still want to know how many total rows satisfied the criteria. The MySQL function, found_rows(), can do such a thing without requiring you run a second full query against the database. Here’s an example: =sqlexec(m.liH, "select sql_calc_found_rows * from BIZ where cnabiz = 'A%' limit 100", "csrJustA") =sqlexec(m.liH, "select found_rows()", "csrHowMany")

This very topic was the subject of a recent user group meeting discussion, as one member recounted how an application he was working on did an initial select against the table to determine how many rows were in it, then a second select to bring down all of the data in the table, and then a third query against the result to filter down to the results they were interested in. Yes, this was an application in dire need of some performance tuning, or as our storyteller suggested, some rewriting from scratch.

Using auto-increment to handle primary key s Most Fox developers have had the occasion to roll their own mechanism for handling primary key generation. One such mechanism was to use a table to hold the last used primary key for each table; the routine to insert a new record would lock the appropriate record in this table, update the last used key value, unlock the record, and return the new ‘last used’ key value as the primary key value to use for the insert. It is a relief, then, to have this work handled automatically; by marking a MySQL table’s primary key field as “auto-increment,” inserts automatically have a new primary key value assigned by the database engine. This means SQL inserts do not even have to reference the primary key field any more. Note, however, that the primary key value generation can be overridden manually, by explicitly providing a value for it, like so: insert into BIZ (iidbiz, cnabiz, cnasec, cadded, tadded) values (219, 'A New Biz', 'Row 2', 'bob', now())


Naturally, the primary key value you explicitly provide must satisfy the primary key constraints – it can’t already be used for another record, for example. It is very comforting to know that MySQL watches over your shoulder and keeps track of the largest primary key value you use. This way, if you should then turn the primary key generation back over to the MySQL engine, it won’t attempt to re-use a value you manually inserted. For example, suppose the BIZ table had primary keys automatically generated in the range of 1 to 1200, and then you inserted a few rows with values from 1326 through 1331. If you then had MySQL generate the next primary key value, it would use 1332.

Determining the last value of an auto-incremented field The MySQL function, last_insert_id(), returns the last incremented value. This is handy when you do an insert for a parent and then need that value for a child you’re going to insert next. =sqlexec(m.liH, "insert into BIZ (cNaBiz) values ('The Automotive Pizza Parlor')") =sqlexec(m.liH, "select last_insert_id() as cLastID", "csrLastID") m.liLastID = val(csrLastID.cLastID)

While this looks simple enough, in real life you’ll run into a couple of situations that would appear to throw a wrench in the works. First off, what if you insert more than one record with a single INSERT command? Each record will get their own (unique) primary key; what does last_insert_id() do in this case? Answer: last_insert_id() returns the first primary key generated. (Note that this is MySQL specific – other SQL databases may work differently.) Here’s an example: =sqlexec(m.liH, "insert into biz (cnabiz) values ('Automotive Accessories') =sqlexec(m.liH, "select last_insert_id()", "csrLastID") ? csrLastID.last_insert_id 5521 =sqlexec(m.liH, "insert into biz (cnabiz) values ('Bold Bibs'), ('Card Shark'), ('Dropcloth Dungeon'), ('Evergreen Elements') =sqlexec(m.liH, "select last_insert_id()", "csrLastID") ? csrLastID.last_insert_id 5522

The last_insert_id() value is 5522, corresponding to ‘Bold Bibs’. If you looked at the table, you’d see that ‘Card Shark’ was assigned 5523, ‘Dropcloth Dungeon’ is 5524, and 5525 is Evergreen Element’s primary key. (This all assumes that only one user is inserting records, as discussed shortly.) Also note that the primary key value generated is for the current connection, and is handled by the database engine. In other words, MySQL takes care of multi-user issues with respect to generating unique primary key values and keeping straight which one goes to which user. Suppose two separate clients (using different connections) were both inserting records. Before they both began, the last primary key value was 1006. Then Alice inserts a record and calls last_insert_id(). At nearly the same time, Bob inserts three records and also calls last_insert_id(). Alice sees 1007 in the return of last_insert_id() while Bob, a blimptosecond later, sees 1008. Then Carl inserts a record, and he sees 1011 as his most recent primary key value. Alice’s value of 1007 can’t be affected by the work that Bob and Carl do, and vice


versa for their last_insert_id() function calls as well. Everything is taken care of as one would hope and expect. The possibility of two users inserting records, and MySQL handling the key generation, means you can’t just assume that a multiple record insert by one user would use a contiguous set of primary keys. In other words, just because last_insert_id() returned ‘5522 for ‘Bold Bibs’ doesn’t mean that Evergreen Element’s primary key had to be 5525. The MySQL online documentation lists a couple of caveats regarding the last_insert_id() function that you’ll want to read, particularly in the areas of errors and transactions.

Conclusion/Summary While navigating through a system via a client-server paradigm is more difficult than with a traditional LAN application, adding/editing and deleting is actually easier. No more multi-user contention issues; the back end takes care of it all for you. In fact, the engine can do even more, as we'll see in the next couple of chapters. Updates and corrections to this chapter can be found on Hentzenwerke’s Web site, www.hentzenwerke.com. Click “Catalog” and navigate to the page for this book.


Chapter 20: Relational Integrity 369

Chapter 20 Relational Integrity Most Visual FoxPro developers are familiar with the concepts of relational integrity, and many have at least dabbled with having the database enforce RI instead of handwriting custom code to do so. In this chapter, we’ll add RI to one of our sample application’s forms.

We looked at the mechanisms for enforcing relational integrity through foreign key constraints back in Chapter 10, “Creating Data Sets from Scratch,” in the Assigning foreign keys section. But that section just showed you how to swing the hammer, no mention of nails or two-byfours anywhere. How does the foreign key mechanism work with application code? We’ll set up foreign key constraints on the INS database, and then show how to take advantage of the RI that’s now sleeping under the hood when we work with the interface.

Revisiting the interface We’re going to use the “All-in-One” Add/Edit/Delete form that we created in Chapter 19, shown again in Figure 1, and add a delete restriction to some of the child tables that will fire when a parent (business) record is deleted.

Figure 1. The "All-in-One" form from Chapter 19 is going to get some automatic delete capabilities.

370


The form was initially designed to do the following when a business was deleted: • • •

•

Check to see how many locations are associated with the business, If zero locations are associated with the business, delete the biz record, If only one location is associated with it, delete the coordinates, persons, and the location, the biz types, and, finally, the biz record, and If more than one location is associated with the business, no deletion is allowed.

The code in the deletebiz() method looks (roughly) like this: do case case m.liHowManyLoc = 0 * no loc to delete * delete the biz types thisform.deletebizcat() case m.liHowManyLoc = 1 * delete the coordinatess thisform.deletecoor() * delete the persons thisform.deleteperson() * delete the sole loc thisform.deleteloc() * delete the biz types thisform.deletebizcat() otherwise * lots of locations, so we don't delete locations * AND we don't delete business return .f. endcase * if m.liHowManyLoc = 0 or 1, continue processing... * and delete the business!

The trouble is that this is all hand-coded, which lends itself to errors. Even worse, this business logic is now trapped in this form; the next form or application that comes along will need to have this logic replicated, which potentially leads to even more errors. Additionally, if someone uses this database from another vehicle, such as an Access query, they could delete selected records and leave orphans or missing intermediary records. Some of this logic is just a natural for being taken care of via relational integrity; see Chapter 21, “Stored Procedures,” for an idea of how the rest of the logic can be handled via stored procedures. Specifically, if a Location is deleted, the Coordinates and Persons associated with that Location can be automatically deleted. And if a Business is deleted, the BizCats can be deleted automatically. The new logic (without Stored Procedures) would look like this: do case case m.liHowManyLoc = 0 * no loc to delete * going to delete the biz types automatically case m.liHowManyLoc = 1


* * * *

going to delete going to delete going to delete delete the sole

the biz types automatically the coordinates automatically the persons automatically loc

•

thisform.deleteloc() otherwise * lots of locations, so we don't delete locations * AND we don't delete business return .f. endcase * if m.liHowManyLoc = 0 or 1, continue processing... * and delete the business, which hits biztypes too!

Creating foreign key constraints In order to take care of these rules automatically, we need three RI rules: • • •

parent = loc, child = coor: if parent (loc) deleted, delete the children (coor) parent = loc, child = person: if parent (loc) deleted, delete the children (person) parent = biz, child = bizcat: if parent (biz) deleted, delete the children (bizcat)

Each time one of the parents is deleted, the RI rule causes the child records to be deleted automatically. (These are cascading deletions, as opposed to restricted deletions, where the deletion of a parent would be prohibited if it had children.) As a result, the thisform.deleteperson() method no longer needs to be explicitly called when a business is deleted. (It still needs to exist, because a user might choose to simply delete a person.) Here’s how to set up these RI rules. The one we’re going to walk through step-by-step is the LOC-COOR relationship. Open the Query Browser, select the INS database, right-click on the COOR table, and select “Edit” from the context menu. The MySQL Table Editor displays, as shown in Figure 2.

372


Figure 2 . Opening the COOR table in the MySQL Table Editor. We’re going to add a foreign key to the COOR table that indicates a COOR record should be deleted when the corresponding parent record (in the LOC table) is deleted. Click the Foreign Keys tab in the bottom of the dialog, so your table editor looks like the one shown in Figure 2. Click the “+” sign under the listbox on the left side of the Foreign Keys tab, bringing forward the Add Foreign Key dialog, shown in Figure 3.

Figure 3 . MySQL creates a default value for the new foreign key. Change the Foreign Key Name to “FK_iidloc,” as shown in Figure 4.


Figure 4 . Changing the suggested name to something more friendly. After clicking OK, you’ll see the foreign key added to the list box in the Foreign Key tab. Next, identify the Ref. Table, which in this case is the LOC table. Finally, define what the On Delete action will be. In this example, change On Delete from “Restrict” to “Cascade.” All three of these changes are shown in Figure 5.

Figure 5 . Results of adding a foreign key. Once you click the Apply Changes button, you’re prompted to confirm your changes, as shown in Figure 6.

374


Figure 6 . Confirmation of command to add the foreign key. You can choose to cut and paste the SQL statement out of the edit box and execute it manually if you like. Note that if you already have an RI foreign constraint defined on a field, using the Table Editor to change an attribute of that constraint will likely fail (due to a bug in the way MySQL currently handles RI). You’ll need to delete the old constraint and add a new one from scratch.

Using foreign key constraints It’s probably best to try a few deletes interactively before adding the complexity of a VFP form.

Using RI interactively Open the Query Browser and add a new BIZ record: insert into biz (cnabiz) values ("Wild Life of Hollywood Insurance")

Take note of the new iidbiz in BIZ (you could do a select * from BIZ

or just use last_insert_id() as discussed at the end of Chapter 19). Next, add a new LOC for this new BIZ: insert into loc (iidbiz, cno, edir, cstreet, csuf) values (504, ' 9876', 'W', 'Main', 'St')

where 504 was the new iidbiz for Wild Life of Hollywood (your iidbiz might be different). Similarly, find the iidloc primary key for the LOC record just added, in this case, 514. Finally, add a coordinate or two for this location:


insert into coor (iidloc, cdata, etype) values (514, '201-555-1212', 'voice') insert into coor (iidloc, cdata, etype) values (514, '800-555-1212', 'toll-free')

So you have a new BIZ (“Wild Life of Hollywood Insurance”), a new LOC for that BIZ (on Main Street), and two COOR (voice and toll-free phone numbers) for the new LOC. According to the new RI rule you just set up, deleting the LOC should automatically delete the two COOR records, right? Give it a shot (substituting the appropriate primary key in your table, of course.) First, verify the record you want to delete is there: select * from LOC where iidloc = 514

This should show the Main Street record. Easy come, easy go – now get rid of it: delete from LOC where iidloc = 514

Finally, check out what’s what in COOR: select * from COOR

You should see no records for the new location.

Using RI in a VFP form Once you have your RI rules built and running correctly in MySQL, it’s time to try them out with VFP in charge. This step is kind of anti-climactic, as all it requires is changing the code in the deletebiz() method described earlier, and optionally, adding the W ild Life of Hollywood’s location and coordinate records. (The source code for this chapter has the new version of the ui_allinone.scx form.) Then run the form, navigate to the W’s, select “Wild Life of Hollywood Insurance” in the list box, and click the Delete button next to the Main Street address. The coordinates should be gone.

Errors encountered during RI setup In a perfect world, well, stop me if you’ve heard this one before. While it all looks so easy as you’re reading along, as soon as you get your fingers on the keyboard, something is bound to fail. You add the foreign key, select the On Delete and On Update conditions, choose the columns, and hit “Apply Changes.” And voila, a dialog like that shown in Figure 7 appears. It sure happens to me often enough. Oh, what to do?

376


Figure 7 . A sample error message encountered when creating a foreign key. Here are a few things that could potentially trip you up.

InnoDB only! The first problem is so sneaky that it doesn’t even throw an error. You can construct an RI constraint with a MyISAM table, click the “Apply changes” button in the Table Editor, and no error message will display, but the constraint will not be saved. The next time you open the Table Editor, the constraint you so painstakingly created will be gone. Why? RI only works with InnoDB tables. If you defined your tables as MyISAM, no dice, Charlie.

Indexes needed on both sides of FK expression The “Can’t create table” error message isn’t very handy. The first few times you run into it, you’re bound to dig around on disk, looking to see if you ran out disk space or if you have permissions problems. This message is a red herring, though. Indexes are needed on both sides of the foreign key expression. Here’s a concrete example. Suppose you’re creating a foreign key constraint between BIZ and BIZCAT, using the ‘iidbiz’ foreign key field in BIZCAT and the ‘iidbiz’ primary key field in BIZ. You’d most likely create indexes on both of those key fields out of habit because you’d have done so with VFP so as to take advantage of Rushmore optimization. With MySQL, though, it’s not just a good idea; you are required to create them in MySQL in order to get foreign key constraints to work.

Index fields need to be same type, and maybe length Index fields must be the same data type, in order to avoid data conversion. You can imagine how inefficient it would be to have to convert an INT field to a CHAR data type when relating two tables. Furthermore, while CHAR fields don’t have to be the same length, INTEGER fields do – you can’t have a primary key of type INTEGER and a foreign key field in another table of type INT(10).


Constraint name needs to be unique When you start to create a foreign key, you’re prompted for a constraint name, as shown back in Figure 3. While MySQL offers to create a name for you, I’ve advocated using your own, more descriptive names so you can readily identify which foreign key is which. There is a hidden danger here, though, in that the constraint name must be unique – throughout the database! If you try to create “FK_iidbiz” in the BIZCAT table and then again in the LOC table, you’ll get a “150” error similar to that shown in Figure 7. Sad to say (to me), but if you’re creating lots of constraints, you might be better off letting MySQL name them all for you.

Conclusion/Summary RI is an important capability in MySQL, but there are a number of subtle conditions you need to be aware of. Check out the MySQL documentation for Foreign Key Constraints (currently in section 14.2.6.4) at http://dev.mysql.com/doc/refman/5.0/en/innodb-foreign-key-constraints.htm Once you get past the internal leap-of-faith issue of letting RI take care of your table relationships, you may wonder why you ever hand-wrote code to cascade or restrict deletes and updates. Updates and corrections to this chapter can be found on Hentzenwerke’s Web site, www.hentzenwerke.com. Click “Catalog” and navigate to the page for this book.

378


Chapter 21: Getting Started with Stored Procedures 379

Chapter 21 Getting Started with Stored Procedures MySQL 5.0 made news with the addition of stored procedures, a feature that many claimed prevented MySQL from being one of ‘the big boys’ in the database server arena. In this chapter, we’ll explore how MysQL’s stored procedures work and how to use them from within Visual FoxPro.

Stored procedures have been around forever (or what seems to be forever) in some database systems, but to some VFP developers they are a brand new experience. They enable you to bind program code directly to data so business logic and data validation stays with the data, independent of the application that accesses the data. Indeed, some swear by stored procedures; one reviewer of this book describe how in his systems, “No one ever touches the data directly. All data access is routed through stored procedures.” While that level of sophistication is beyond the scope of this book, it’s a concept worth keeping in the back of your mind.

What is a stored procedure? A stored procedure is simply a chunk of code stored in the database instead of being part of the application. Like VFP subroutines, there are two kinds – procedures, which perform a process, and functions, which do processing and then return a value. In both cases, stored procedures are part of the MySQL database, not part of your application. Philosophically, you approach a stored procedure just like you do the data in a database – it belongs to the database administrator, not to you, the author of an external application.

Creating stored procedures in MySQL Let’s look at building stored procedures in MySQL first. Once we’re comfortable creating and calling them from within MySQL, we’ll learn to call them from VFP.

Make sure MySQL is set up for stored procedures Since stored procedures were added to MySQL in version 5, you need to make sure your version and your data is enabled and set up. It’s best to do these things in the MySQL monitor, because some of them can’t be done in the command pane of the Query Browser. Using the command prompt in Windows, do the following: c:> mysql -u whil -p Enter password: ****** Welcome to the MySQL monitor. Commands end with ; or \g mysql>

The monitor is all set and ready to go.

380


Step 1. Make sure you’re running version 5 Use the “Show variables” command, like so: mysql> show variables like 'version';

and press Enter at end of line to execute. You’ll see something like: +----------------+-----------------------+ | Variable_name | Value | +----------------+-----------------------+ | version | 5.0.21-community-nt | +----------------+-----------------------+

This command also works in the Query Browser.

Step 2. Make sure mysql.proc table exists While you can investigate the existence of the proc table via a SELECT, like so: mysql> use mysql; Database changed mysql> select db, name, type from mysql.proc; +-------+-------+-----------+ | db | name | type | +-------+-------+-----------+ | db4sp | p1 | PROCEDURE | | db4sp | p2 | PROCEDURE | | db4sp | p3 | PROCEDURE | | db4sp | p4 | PROCEDURE | | db4sp | p5 | PROCEDURE | +-------+-------+-----------+ mysql>

it might be easier to fire up the Query Browser and drill down through the Schemata tab.

Step 3. Change the delimiter MySQL’s default delimiter is the semi-colon “;”. However, when you’re using stored procedures that are longer than a single line, you may find yourself wanting to use a semicolon to terminate a single statement. Thus, you need a different delimiter to terminate the entire stored procedure creation command. You can use the ‘delimiter’ command to do so: mysql> delimiter | mysql> show databases|

Now let’s try our new delimiter out: mysql> show databases| +--------------------+ | Database | +--------------------+ | information_schema | | mysql | +--------------------+ mysql>


Different folks prefer using different characters for a stored procedure delimiter; the goal, whichever you choose, is to select one that won’t be used in the body of a stored procedure. Some folks argue against a pipe, as it’s conceivable that you could use a || construct in a stored procedure. Conceivable, yes, but not probable at this desk. You can change the delimiter back like so: mysql> delimiter ;

Note that you have to change the delimiter each time you enter the monitor. You could modify your MySQL config file to make the change permanent, but you probably don’t want to. The point of setting the delimiter to something different is so you can use the “normal” semi-colons inside the SP. It’s like embedding quotes and square brackets. If you reset it globally, you still need to set it at the beginning of an SP/trigger so you can tell the difference between the individual line-endings and the end of the SP declaration. The delimiter command does not work in the Query Browser.

Create your very own stored procedure Hope you’re sitting down, cuz this is going to get pretty exciting right around the corner. The first SP we’re going to create is a function. It will return ‘Hello World’, as all good first programs should. However, before we actually create the procedure, we need to create a database that it will be associated with. We’ll also create a table inside the database, just so we have something to monkey around with. Issue the command: mysql> create database db4sp| Query OK, 1 row affected (0.10 sec) mysql> create table data4sp (iiddata4sp int, cnaf char(25))| Query OK, 0 rows affected (0.25 sec) mysql>

We now have a database, db4sp, with a table, data4sp. Now throw a row or two into the table: mysql> insert into data4sp (iiddatat4sp, cnaf) values (1, 'al')| mysql> insert into data4sp (iiddatat4sp, cnaf) values (22, 'barbara')| mysql> insert into data4sp (iiddatat4sp, cnaf) values (333, 'carl')|

Good enough. Now let’s create the stored procedure. Issue the command: create procedure p1() select 'Hello World';|

and press Enter after typing the “|”. Your MySQL monitor should look like this: mysql> create procedure p1() select 'Hello World';| mysql> delimiter ;

Let’s explain the pieces. “create procedure” is a command similar to “create database,” and, indeed, it does something similar.

382


“p1” is the name of the proc, and the “()” after it are the parameter list. Normally, you’d probably want to name your stored procedures with more useful names, but this will work as a start. As you can tell, there aren’t any parameters in this procedure, but you still have to include the parentheses, empty as they may be. We’ll add parameters in the next example. “select ‘Hello World’” is the body of the stored procedure – the program code, as it were. This is what is going to be executed when we run the stored procedure. If the syntax looks funny, recall back to Chapter 17, where I mentioned that “select now()” in the Query Browser’s command pane (or the MySQL monitor) was a cheap equivalent to VFP’s command window for getting quick interpretation of commands. “Select ‘Hello World’” simply returns a character string instead of interpreting a function. (Some of my reviewers felt this use of SELECT was a bit contrived, and I agree, but we have to get that traditional “Hello World” call in somewhere.) This syntax works with MySQL, but not with VFP because VFP thinks ‘Hello World’ is the name of a table alias, due to backwards compatibility when ‘select’ was used to switch between work areas. The “;” indicates that this is the end of this particular SQL command (a stored procedure can have multiple SQL commands, but they each need to be terminated with a “;”). The “|” terminates the entire stored procedure, since we just changed the MySQL delimiter in the previous section. Remember to reset the delimiter immediately back to the default semi-colon or you’ll get confused later on. IMPORTANT POINT: This stored procedure is associated with the db4sp database, because that was the database open when we created the stored procedure. This is an important point, and one that can bite you if you’re not paying attention, so I mention it right away. I’ll explain more about the internals in a few pages, for now let’s move on to running the stored procedure.

Run the proc from the MySQL monitor Nothing fancy here! Just use the ‘call’ statement and the name of the stored procedure: mysql> call p1(); +--------------------+ | Hello World | +--------------------+ | Hello World | +--------------------+ 1 row in set (0.00 sec) mysql>

OK, so we’ve seen that we can create and call a stored procedure that returns the string “Hello World.” This isn’t a very useful procedure, nor does it have anything to do with data. However, it is a procedure in the database, not a routine in your own VFP application, or anywhere else for that matter, and that’s what counts right now. What this means is that your VFP program (as well as your PHP program and your C program and your Python program) can access and call this stored procedure as long as it can connect to the database. (Actually, you can configure stored procedures with rights to allow them to only be called by selected users, but that’s beyond this discussion.) Now let’s create a procedure that has something to do with data.


Create a procedure that accesses a database table First, let’s remind ourselves of the statement we want to execute in the stored procedure. mysql> select * from data4sp; +------------+---------+ | iiddata4sp | cnaf | +------------+---------+ | 1 | al | | 22 | barbara | | 333 | carl | +------------+---------+ 3 rows in set (0.00 sec) mysql>

Now let’s create the procedure that performs this SELECT: mysql> delimiter | mysql> create procedure p2 () select * from data4sp; | Query OK, 0 rows affected (0.40 sec) mysql> delimiter ;

And, finally, let’s execute the procedure: mysql> call p2(); +------------+---------+ | iiddata4sp | cnaf | +------------+---------+ | 1 | al | | 22 | barbara | | 333 | carl | +------------+---------+ 3 rows in set (0.00 sec) mysql>

This is an example of calling a stored procedure that is just a procedure. It does something but doesn’t return a value, as opposed to a function that returns a value (such as calling pi() returns 3.14159). Exercise for the reader? Create a stored procedure that selects just records with a primary key > 10.

A stored procedure with a parameter Let’s move on to creating a procedure that takes a parameter. mysql> create procedure p3 (tnow datetime) select tnow;| mysql> delimiter ;

You’ll see in the parentheses that you need to name the parameter and identify its data type. I used Hungarian naming for the parameter name, but you can call it anything you like. Now call the procedure, passing a parameter to the procedure:

384


mysql> call p3(now()) +----------------------+

| tnow | +----------------------+ | 2007-06-15 15:18:44 | +----------------------+ 1 row in set (0.00 sec) mysql>

Just for fun, let’s now call it incorrectly – without any parameters: mysql> call p3(); ERROR 1318 (42000): Incorrect number of arguments for PROCEDURE db4sp.p3; expected 1, got 0

One of the greatest error messages I’ve ever seen – it tells you what you did wrong, where you did it, and what happened. And now let’s call it with a parameter – of the wrong data type: mysql> call p3('today'); ERROR 1292 (22007): Incorrect datetime value: 'today' for column 'tnow' at row 1 mysql>

Passing a parameter, somewhat more involved Let’s create a somewhat more complicated procedure with a parameter: mysql> create procedure p4(tnow datetime) select *, 'Hello World', tnow from data4sp;| mysql> delimiter ; mysql> call p4(now()) ; +------------+----------+-------------+---------------------+ | iiddata4sp | cnaf | Hello World | now() | +-------+---------------+-------------+---------------------+ | 0000000001 | al | Hello World | 2007-06-18 22:50:42 | | 0000000022 | barbara | Hello World | 2007-06-18 22:50:42 | | 0000000333 | carl | Hello World | 2007-06-18 22:50:42 | +------------+----------+-------------+---------------------+ 3 rows in set (0.00 sec) mysql>

Using a parameter more constructively Let’s combine everything we’ve done, using that parameter in a more productive manner, as part of a WHERE clause in the SELECT statement. mysql> create procedure p5 (tid int) select * from data4sp where iid = tid;| mysql> call p5(2)| Empty set (0.02 sec) mysql> call p5(22)| +------------+----------+ | iiddata4sp | cnaf | +-------+---------------+ | 0000000022 | barbara | +------------+----------+ 1 rows in set (0.00 sec) mysql>


Notice that the call to the procedure works when you pass a value that is legal, but results in no records being returned.

Using input and output parameters MySQL stored procedures can use both input and output parms. An output parm is what you and I would call a parameter passed by reference – once the routine is finished the output parm contains a value that will be visible outside of the procedure. Here’s how to create a procedure that uses both input and output parms. Pay attention, because we’ll use this again later in this chapter. This stored procedure will take an input value (iid), multiply it by 2, and return the result (i). mysql> delimiter | mysql> create procedure p6(iid int, out i int) -> begin -> set i=iid*2; -> end;| Query OK, 0 rows affected (0.00 sec) mysql> delimiter ;

Note the use of the “;” to separate statements in this multi-line procedure, and then the use of the “|” delimiter to tell MySQL we’re done with the “create procedure” statement. Now let’s call it. You’ll notice that we need to execute two commands. The first calls the stored procedure, passing the input value, 123, and a placeholder for the return value, i. Calling the procedure generates a result in the variable “i”. Then we have to determine what “i” is, via a second command (the SELECT). mysql> call p6(123,@i); Query OK, 0 rows affected (0.00 sec) mysql> select @i;| +-----+ | @i | +-----+ | 246 | +-----+

Cleaning up After you’re done experimenting, you may want to clean up after yourself. The Query Browser has a number of useful tools for working with stored procedures. Click the Schemata tab and select the database you’re working with. Stored procedures display in a list along with tables, albeit with a flow-chart-like diagram, as shown in Figure 1.

386


Figure 1. Viewing stored procedures associated with a database in the Query Browser’s Schemata tab. Stored procedures with parameters have a black arrow to the left of the icon; clicking it will display what the parameter list looks like. Right-clicking on a stored procedure displays context menu items similar to those you’re used to seeing for tables: • • •

Edit procedure Drop procedure Copy SQL to clipboard

Selecting “Edit procedure” opens the highlighted procedure in the main Query Browser window, as shown in Figure 2. Much handier than trying to hand-craft a long routine in the MySQL monitor.


Figure 2 . The Query Browser has procedure editing capability. There are command equivalents to the functionality provided by the Query Browser. You get rid of a procedure via the command ‘drop procedure p1’, and ‘alter procedure’ allows you to edit an existing procedure. As with ‘alter table’, the syntax is exacting and takes some practice.

Where is a stored procedure stored? Stored procedures are stored in the routines table of the information_schema database. You can view the stored procedures via a SELECT command against information_schema.routines. The following command displays the name and body of all stored procedures in all databases. mysql> select routine_schema, routine_name, routine_definition from information_schema.routines +----------------+--------------+------------------------------+ | routine_schema | routine_name | routine_definition | +----------------+--------------+------------------------------+ | db4sp | p1 | select 'Hello World' | | db4sp | p2 | select * from data4sp | | db4sp | p3 | select tnow | | db4sp | p4 | select 'Hello World', tnow | | db4sp | p5 | select * from data4sp whe... | +----------------+--------------+------------------------------+ 5 rows in set (0.00 sec) mysql

The routine_schema field contains the name of the database a stored procedure is associated with. You can also use ‘show procedure’ (similar to ‘show databases’) but, while shorter and thus handy, it’s a MySQL specific command, and additionally, doesn’t provide quite as much information as a SELECT.

Calling a MySQL stored procedure from VFP To regroup, we have a database, db4sp, with a table, data4sp, that contains a single field and several records. We also have five stored procedures: p1: p2: p3: p4: p5:

select select create create create

'Hello World' * from data4sp procedure p3 (tnow datetime) select tnow procedure p4(tnow datetime) select 'Hello World', tnow procedure p5 (tid int) select * from data4sp where iid = tid

388


We’re going to create a series of VFP programs that walk through the use of these stored procedures (well, not all of them, as some are pretty similar). We’ll piggyback on the Z_SQL.PRG procedure file that we used a few chapters ago, but the meat of our work will be contained in programs named CH21A.PRG, CH21B.PRG, and so on. These programs are similar to the routines in Chapter 12 through 14. A copy of Z_SQL.PRG is contained in the source code for this chapter, so you don’t have to go scurrying about for source code files from earlier chapters.

Calling your basic stored procedure from VFP Calling a SP from within VFP uses SQLEXEC(), just like we’ve been using in the rest of this book. (In fact, you have to use SQLEXEC(), since stored procedures can’t be called from remote views.) The essential code to call a stored procedure from within VFP looks like this: m.lcStr = "call p1()" m.lcStrErr = m.lcStrErr ; + iif(z_sqlexec(m.liH, m.lcStr), "", "Command '" + m.lcStr + "' failed." ; + chr(13))

where z_sqlexec() is our wrapper for VFP’s native sqlexec(), returning .t. if sqlexec() is successful and .f. if not. If you plunk these two lines into a PRG after creating a m.liH handle via SQLSTRINGCONNECT, you’ll run into an error. Looking at ZOOPS, the text of the error message (via our friend AERRORS()) is Connectivity error: [MySQL][ODBC 3.51 Driver][mysqld-5.0.21-communitynt]PROCEDURE .p1 does not exist

This is a tough one to puzzle out, since the p1 procedure very obviously exists – we’ve been calling it from the MySQL monitor for the last half hour. The reason VFP (and MySQL) can’t find it is because it’s associated with the db4sp database, which VFP doesn’t know anything about yet (unless you included a reference to the database in your ODBC connection). When we were in the MySQL monitor, one of our first steps was opening the database. Just because the database was open in the MySQL monitor doesn’t mean VFP knows it’s open. Ya gotta ‘use’ the db4sp database from within VFP before using its stored procedures! Here are the salient portions of the code needed: * attach to the database for these stored procs m.lcStr = "use db4sp" m.lcStrErr = m.lcStrErr ; + iif(z_sqlexec(m.liH, m.lcStr), "", "Command '" + m.lcStr + "' failed." ; + chr(13)) * call the first stored proc m.lcStr = "call p1()" m.lcStrErr = m.lcStrErr ; + iif(z_sqlexec(m.liH, m.lcStr), "", "Command '" + m.lcStr + "' failed." ; + chr(13))


This code is found in CH21A.PRG. You might want to take a look at CH21A.PRG as there are a couple of enhancements to the surrounding elements since the last time we used it in earlier chapters.

Passing a hard-coded parameter to a stored procedure Now let’s add some more to CH21A.PRG and call it CH21B.PRG. The goals in this version are to call a stored procedure from VFP that includes a parameter, and to control the result set created by the stored procedure’s SELECT statement. The z_sqlexec() procedure has code that allows you to explicitly pass the name of a cursor you want created (instead of using VFP’s default); in the following listing, you’ll see “csr_p1” and “csr_p4” being passed. * attach to the database for these stored procs m.lcStr = "use db4sp" m.lcStrErr = m.lcStrErr ; + iif(z_sqlexec(m.liH, m.lcStr), "", "Command '" + m.lcStr + "' failed." ; + chr(13)) * p1: call the first stored proc m.lcStr = "call p1()" m.lcStrErr = m.lcStrErr ; + iif(z_sqlexec(m.liH, m.lcStr, '', 'csr_p1'), ; "", "Command '" + m.lcStr + "' failed. " + chr(13)) * p4: call a stored proc with a parm m.lcStr = "call p4(now())" m.lcStrErr = m.lcStrErr ; + iif(z_sqlexec(m.liH, m.lcStr, '', 'csr_p4'), ; "", "Command '" + m.lcStr + "' failed. " + chr(13))

The other advantage I snuck in there is that by explicitly naming the cursors you can now examine the results of multiple stored procedures instead of each stomping on the cursor created in the previous procedure.

Passing a user-provided parameter to a stored procedure We’re going to modify CH21B.PRG, calling it CH21C.PRG, and incorporate a user-supplied parameter to the stored procedure. This will be procedure p5 that filters the SELECT statement via a user-supplied parameter. We’ll pass the parameter, the id value, via an lparameters statement: lparameters m.tcUN, m.tcPW, m.tiID

and the program will be called like so: do CH21C.PRG with 'bob', 'secret', 123

where 123 is the ID value. The other interesting code in CH21C looks like this: * p5: call a stored proc with a parm m.lcStr = "call p5(" + alltrim(str( m.tiID )) + ")"

390


m.lcStrErr = m.lcStrErr ; + iif(z_sqlexec(m.liH, m.lcStr, '', 'csr_p5'), ; "", "Command '" + m.lcStr + "' failed. "+chr(13))

Note how the ID value, m.tiID, is concatenated with the rest of the ‘call’ statement, so the statement sent to MySQL looks like this: call p5(123)

Also note that the result is returned in a new cursor, named ‘csr_p5’, so this call can be included in CH21B.PRG and the results of each stored procedure can be examined.

Incorporating stored procedures in the sample app In Chapter 20 I promised a look at how to use stored procedures to encapsulate business logic, such as rules for deleting businesses (and the corresponding child records). The sample app built in the previous chapters use a series of routines to contain business rules. There’s no reason why that business logic can’t be moved into a set of corresponding stored procedures, and then have those procedures called from the same places in the VFP form. For instance, let’s take a look at the business logic in deletebiz(). One part of the code requires the knowledge of how many locations exist for the current business. This requirement could easily be turned into a function – HowManyLocForThisBiz(iidbiz). We’ll again need to pass input and output parameters: mysql> create procedure howmanylocforthisbiz(iid int, out ihowmany int) -> begin -> declare ii int; -> select count(*) from loc where iidbiz = iid into ii; -> set ihowmany = ii; -> end;| Query OK, 0 rows affected (0.00 sec) mysql>

This time, when we pass a value to the stored procedure, we’re passing a primary key to the BIZ table, which in LOC identifies which BIZ the LOC belongs to. mysql> call howmanylocforthisbiz(123,@ihowmany)| Query OK, 0 ro... mysql> select @ihowmany| +-----------+ | @ihowmany | +-----------+ | 1 | +-----------+

Now, how to call that from VFP? CH21D.PRG contains the full routine to do so. Here’s the relevant part. The first call executes the howmanylocforthisbiz stored procedure: * call the proc, creating a result set in csr_p6.i m.lcStr = "call howmanylocforthisbiz(" + alltrim(str( m.tiID )) ; + ", @iHowMany)" m.lcStrErr = m.lcStrErr ;


+ iif(z_sqlexec(m.liH, m.lcStr, '', 'csr_p6'), ; "", ; "Command '" + m.lcStr + "' failed. " ; + chr(13)) * determine what the value in the result set is m.lcStr = "select @i as iHowManyLoc" m.lcStrErr = m.lcStrErr ; + iif(z_sqlexec(m.liH, m.lcStr, '', 'csr_p5'), ; "", ; "Command '" + m.lcStr + "' failed. " ; + chr(13)) messagebox("Number of locations for biz ID # " + transform(m.tiID) ; + "-> " + alltrim(transform(csr_p5.iHowManyLoc)) + " <-")

Finally, try calling howmanylocforthisbiz() from within the deletebiz() method in the ui_allinone.scx form, and using the return value to continue on with the rest of the method. * call the proc, creating an output parm m.lcStr = "call howmanylocforthisbiz(" + alltrim(str( m.liidBiz )) + ", @i)" m.liX = sqlexec(thisform.iH, m.lcStr) if m.liX > 0 * now create the result set that contains the output parm, and * then determine what the value in the result set is m.lcStr = "select @i as iHowManyLoc" m.liX = sqlexec(thisform.iH, m.lcStr, 'csr_p6') *\\\ could delete this once you're comfortable it's working! messagebox("Number of locations for biz ID # " + transform(m.liidBiz) ; + "-> " + alltrim(transform(csr_p6.iHowManyLoc)) + " <-") m.liHowManyLoc = csr_p6.iHowManyLoc else local aOops[1] m.liHowManyErrors=AERROR(aOops) =thisform.ocslib.z_sqlerror(@aOops, m.lcStr, m.liHowManyErrors) * we don't know how many others there are, so assume there is more than one m.liHowManyLoc = -1 m.liidLoc = 0 endif

See ui_allinone.scx in this chapter’s source code for the entire method.

Conclusion/Summary So that’s the mechanics of calling a stored procedure from VFP. There is a lot more to stored procedures than this introduction, but I’ve already covered all of the information that’s specifically relevant to Visual FoxPro. If you want to learn more, check out Peter Gulutzan’s 60 page guide on MySQL 5.0’s Stored Procedures. At this writing, this is found at dev.mysql.com/tech-resources/articles/mysql-storedprocedures.htm

Whether you choose to use stored procedures or not, understanding how they work and how to incorporate them in your applications is, with MySQL 5.0, an important skill to possess. Updates and corrections to this chapter can be found on Hentzenwerke’s Web site, www.hentzenwerke.com. Click “Catalog” and navigate to the page for this book.

392


Chapter 22: Deployment 393

Chapter 22 Deployment As the saying goes, “It’s all fun and games until someone gets hurt.” So it goes with development – it’s also all fun and games until you actually have to make it work in production. Isn’t that someone else’s job? Unfortunately, probably not. In this chapter, we’ll discuss various issues with getting your VFP application up and running in a production environment.

Deployment is a remarkably broad subject, so broad that we published an entire book on it ( Deploying Visual FoxPro Solutions, by Rick Schummer, Rick Borup, Jacci Adams). If you’re responsible for deployment, I highly recommend it – it covers nearly everything you need to know about deploying Visual FoxPro applications in a variety of scenarios, and with a number of different installer tools. Instead of repeating that information during this discussion, I’ll refer you to the appropriate place in that book. Still, as long as that book is – 470 pages – and despite the fact that it has an entire chapter on Client/Server Applications, it doesn’t cover specifics about MySQL, since it uses SQL Server and MSDE as examples. While most of their discussion is applicable, there are still differences with VFP and MySQL, and, as always, the devil is in the differences. This chapter focuses on those differences.

Getting started First, let’s take a look at what you have and where you want to go. The MySQL engine is running on a box, either your own development machine or another machine. Your data is on the same machine as the MySQL engine; that data might be a test database, filled with “Bugs Bunny” and “Daffy Duck” records, or it might be a copy of your actual production database, in order to better test and benchmark your application. On your development machine, which is running Windows, you have the VFP development environment, the MySQL ODBC drivers, and perhaps an ODBC DSN. Your goal is to have a separate production server running the MySQL engine, production data on that machine, and to have a VFP executable (along with runtimes and the MySQL ODBC driver) running on one or (most likely) more Windows workstations. Obviously, there are variations on this scenario – for example you might have your production data on a separate box, and configured the MySQL server engine to point to that other box – but it’s close enough. Note that the issue of what operating system is running on the MySQL server machines – both development and production – hasn’t been mentioned. Why? Because it’s not important – for our purposes, our VFP application doesn’t care what OS is hosting MySQL. So, back to our goal – how do we get there? First of all, since deployment (like data conversion) is one of those tasks rarely performed and under-budgeted, but critical to the success of the system, a checklist is good. Development Build your application's EXE with VFP 

394




Create an installation package for your application, including:  application EXE  VFP runtimes  External reports  help files  active x controls  COM objects  short cuts  config files  temp files  odbc drivers  data source name

Server    

Install production MySQL server Configure production MySQL server user Install production data Open firewall for port 3306

Client  

Test connection to the server Install the application via the installer built on your development machine

Let’s walk through each of these, and discuss what’s new or different in terms of MySQL.

Development Build your application’s EXE with VFP There shouldn’t be anything startling new here; hopefully you’ve been building EXEs occasionally during the development of your application. Doing so will help prevent those last minute errors that can push a project off schedule. For example, the night before delivery is a terrible time to discover you’re using old libraries (or old habits) that still contain references to CTOD and CTOT and you’re compiling with STRICTDATE set to 2.

Create an installation package for your application Once you’ve built an EXE (and it runs properly outside of the VFP IDE), it’s time to build an installer for your application. True, for simple systems you can get away with throwing your EXE and a half-dozen runtimes into a folder and letting your users execute a few manual steps before having a go at it, but you’ll likely find that to be an unsatisfactory long-term solution. Users will want a more polished installation routine, like all of the other software they use, and you’ll get tired of fixing the problems caused by their inevitable errors. In addition to the heavy customization a third party installation tools provide, using a third party tool also serves as a second checklist for making sure you include everything needed by your application and it’s put in the right place. Executables, runtimes, external reports, help files, ActiveX controls and COM objects, shortcuts, configuration files, temp file locations,


and even ODBC drivers and Data Source Names – all of these are components that are set up standard with a third party installer. The Deploying Visual FoxPro Solutions book has over 100 pages with detailed instructions for four popular VFP installers: InstallShield Express, Wise for Windows Installer, InstaFox, and Inno Setup. While you may not need many of the components listed in the previous paragraph, you will need the runtimes and the ODBC driver; these instructions discuss in detail about including them in your installation package. Now that you have an installation package ready for your users, let’s look at the server your application is going to talk to.

Production server Install production MySQL server I’ve already covered the installation of MySQL on both Windows and Linux earlier in this book; by now, hopefully you’ve had the opportunity to do several installs and get comfortable with the ins and outs. My personal bias is to use a Linux machine for the server; I’ve been running both Windows and Linux servers for years, and I’ve found the Linux machines need fewer resources (thus, you can do more with a less powerful machine), are more secure, and need less babysitting. Of course, your choice of operating system is up to you and your customer. One thing you will want to do is turn off as many unneeded services as possible on the server. First, fewer running services means more resources available for programs in use. Once configured, do you really need hardware detection or advanced power management running on a server? Second, fewer running services mean fewer potential entry points for attacks. Do you REALLY want Bluetooth discovery and authentication services running on a server? While you may keep your development machine open, with lots of stuff running on it, the general rule for servers and firewalls is to turn everything off, and only turn on things that you know you need.

Configure the production MySQL server user Now that your production MySQL server has been set up, it’s time to set up permissions so your application can talk to the server on another machine. Little rusty? You need to add a user to your production MySQL database. This user will be the account used by your VFP application when connecting to the database. Properly set up, your application should see no difference between the test database you used during development and the live database your users are accessing in production. The only difference is the connection string to the server. Let’s look at those details now. I would argue vociferously that a feature of your application should include a switch that allows you to switch between data sets. In the olden days when I used DBFs, my development environment folder structure contained separate folders for test, live, and original data. The application was ordinarily pointed to the data in the ‘test’ directory; every so often, once the test data got messed up badly enough, it was refreshed with a pristine copy of test data from the ‘original’ folder. And the ‘production’ folder had a copy of the live data. (The original test data set was often created as a subset of the production data set.)

396


Then, the application had a menu option that allowed the user to switch between test and live data sets. This same mechanism could also be used to switch between multiple data sets if the application needed to be able to do so. In pre-Visual FoxPro days, switching was accomplished essentially by changing the current directory from one data directory to another. With VFP’s data environments being tied to forms, it was a bit more involved, but the concept was the same. With a client-server architecture, providing the ability to switch between datasets becomes easy again. It’s simply a matter of providing multiple connection strings (or DSNs, if you chose to go down that road). For example, the following code uses the MySQL server database on the local machine, with a user of ‘bob’, during development. if m.llInDevelopment m.lcXN = "DRIVER={MySQL ODBC 3.51 Driver};" ; + "SERVER=localhost;PORT=3306;UID=bob;PWD=secret” else m.lcXN = "DRIVER={MySQL ODBC 3.51 Driver};" ; + "SERVER=1.2.3.4;PORT=3307;UID=vfpuser;PWD=superdupersecret” endif

As you see, however, during production the connection string points to another server, on another port, and with a different user. If you go this route, all you need to do is create an account on the production MySQL server named ‘vfpuser’ with a ‘superdupersecret’ password. In your application, of course, you wouldn’t hardcode “bob” or “superdupersecret” in each connection string; rather, you set properties of a globally available application object and refer to those: if goApp.lInDevelopment goApp.cServer = "localhost" goApp.cPort = "3306" goApp.cUsername = "bob" goApp.cPassword = "secret" else goApp.cServer = "1.2.3.4" goApp.cPort = "3307" goApp.cUsername = "vfpuser" goApp.cPassword = "superdupersecret" endif

As a side note, bear in mind that any string embedded in an application (or a DBF or an INI file) is easily read by curious users. If you’re not encrypting user/password information when you store it, you may just want to include no password, and ask each user for it at the start of the session. Another alternative is to provide user credentials that hook you up to the database, where you store the real user passwords you require them to supply to get to the real data. But we’re getting a little far a-field now. Back to the data-set switching technique. And then each connection string would look identical: m.lcXN = "DRIVER={MySQL ODBC 3.51 Driver};" ; + "SERVER=" + goApp.cServer + ";PORT=" + goApp.cPort ; + ";UID=" + goApp.cUsername + ";PWD=" + goApp.cPassword


Still fuzzy? Okay, back to Chapter 5 with you for a refresher.

Install production data Next, you have to get production data installed on the server. There are a couple of possible scenarios here, so let’s walk through each of them. The first is you already have an empty set of database tables that simply need to be installed on the server. Since you already have a test MySQL database to work with during development, it’s likely that you have a VFP program that creates that database. (I did suggest that early on, didn’t I?) As a result, it’s fairly simple to run that program against the production server, creating the database and the empty tables on the production server. Then, as discussed in Chapter 11 and 12, populate those empty tables with available data as needed. No big deal, nothing to see here, move on now. The other possibility is that you already have a production data set ready to go; “all” you need to do is install it on the server. In order to make this happen, you have some options: MySQL’s backup/restore (using the GUI or mysqldump), and direct file copying (using mysqlhotcopy or an alternative). Paul DuBois, in his “MySQL - The definitive guide to using, programming and administering MySQL databases” book, covers each of these in complete detail. So, instead of attempting to duplicate a hundred pages of non-VFP-specific content, I’ll summarize the three and then refer you to his work for more information. The ‘mysqldump’ command simply creates a script (a simple text file) that consists of SQL CREATE and INSERT commands that create your database structure and inserts all of the data into it. See Listing 1 for an example of a part of one such script.

Listing 1. Part of a typical mysqldump script. -- MySQL Administrator dump 1.4 --- ------------------------------------------------------- Server version 5.0.27

/*!40101 /*!40101 /*!40101 /*!40101

SET SET SET SET

@OLD_CHARACTER_SET_CLIENT=@@CHARACTER_SET_CLIENT */; @OLD_CHARACTER_SET_RESULTS=@@CHARACTER_SET_RESULTS */; @OLD_COLLATION_CONNECTION=@@COLLATION_CONNECTION */; NAMES utf8 */;

/*!40014 SET @OLD_UNIQUE_CHECKS=@@UNIQUE_CHECKS, UNIQUE_CHECKS=0 */; /*!40014 SET @OLD_FOREIGN_KEY_CHECKS=@@FOREIGN_KEY_CHECKS, FOREIGN_KEY_CHECKS=0 */; /*!40101 SET @OLD_SQL_MODE=@@SQL_MODE, SQL_MODE='NO_AUTO_VALUE_ON_ZERO' */;

--- Create schema mysql -CREATE DATABASE IF NOT EXISTS mysql; USE mysql; CREATE TABLE `mysql`.`columns_priv` ( `Host` char(60) collate utf8_bin NOT NULL default '', `Db` char(64) collate utf8_bin NOT NULL default '', Ùser` char(16) collate utf8_bin NOT NULL default '', `Table_name` char(64) collate utf8_bin NOT NULL default '',

398


`Column_name` char(64) collate utf8_bin NOT NULL default '', `Timestamp` timestamp NOT NULL default CURRENT_TIMESTAMP on update CURRENT_TIMESTAMP, `Column_priv` set('Select','Insert','Update','References') character set utf8 NOT NULL default '', PRIMARY KEY (`Host`,`Db`,Ùser`,`Table_name`,`Column_name`) ) ENGINE=MyISAM DEFAULT CHARSET=utf8 COLLATE=utf8_bin COMMENT='Column privileges'; CREATE TABLE `mysql`.`db` ( `Host` char(60) collate utf8_bin NOT NULL default '', `Db` char(64) collate utf8_bin NOT NULL default '', Ùser` char(16) collate utf8_bin NOT NULL default '', `Select_priv` enum('N','Y') character set utf8 NOT NULL default 'N', Ìnsert_priv` enum('N','Y') character set utf8 NOT NULL default 'N', Ùpdate_priv` enum('N','Y') character set utf8 NOT NULL default 'N', `Delete_priv` enum('N','Y') character set utf8 NOT NULL default 'N', `Create_priv` enum('N','Y') character set utf8 NOT NULL default 'N',

The MySQL backup facility in the MySQL Administrator (see Figure 1) is merely a front end for this command.

Figure 1. The Backup node in the MySQL Administrator acts as a front end to 'mysqldump'. Click the “New Project” button on the bottom of the dialog, select a database to back up, move it to the Backup Content list using the “->“ arrow, and click “Start backup.” You’ll be asked to choose a location to store the backup script, as shown in Figure 2.


Figure 2 . Selecting a backup file name for the script. There are many options to mysqldump, allowing you to select which tables you want to work with, whether you want to just create the structures or just the data, and how you want to coordinate the load against the server imposed by mysqldump versus the load on the server by users. An advantage to mysqldump is that, since its output is simply a script full of standard SQL commands, you can use the resulting output to move data to another SQL server back end that respects standard SQL. Note that since large databases create large scripts (remember, there’s a separate INSERT command for every row in every table), you will want to acquaint yourself with the options and nuances if database size is a concern of yours. Since MySQL databases are files on disk, you may wonder why you can’t just copy those files to your production server, just like you’re used to copying a folder full of DBFs. And the answer is, you can, but with a couple of caveats. First, if you use native disk copying commands (the DOS ‘copy’ command in W indows or the Linux ‘cp’ and ‘scp’ commands, depending on where the source and target are), you’ll very much want to shut down both the source and target servers while you’re doing a copy, so you don’t try to copy a database file in the midst of the server writing to it. Second, the source and target machines must be compatible, and you will want to make sure you get all of the files. For example, with InnoDB databases, there are .frm files to worry about. DuBois’ book covers the specifics nicely. An alternative to manually copying files yourself is using the ‘mysqlhotcopy’ Perl DBI script. It has all the advantages of manually copying files with the added benefit of coordinating the read/writes with the MySQL server so you don’t have to bring the servers down. Additionally, since it works directly with the file system, it’s faster than mysqldump, which has to go through the database server to create SQL commands. In some situations, there’s an alternative to using backup and restore. It’s sort of cheating, and it’s applicable with a number of caveats, but it’s worth a mention. If you’re using InnoDB and you want to copy all of the databases in your test InnoDB file to your production server,

400


you can simply stop your test and production MySQL servers, copy the “ibdata1” file from your test machine to your production box, and then restart the servers. If you’re using MyISAM files, the caveat about ‘all of your test databases’ doesn’t even apply, as each MyISAM database is in a separate data structure.

Open firewall for port 3306 MySQL communicates through port 3306 by default, so you want to make sure this port on your server’s firewall is open. Many firewall products know the relationship between MySQL and 3306, so they offer an option to open “MySQL” and that will be that. As mentioned in Chapter 6, some reviewers have reported trouble using 3306, and have had to configure MySQL to use 3307 (or another port) instead. If you’re in the same situation, you need to identify that port in your connection string and attend to your firewall as well.

Client machine The final test comes with installing the application and testing it.

Install the application If you used a third party installation tool, you should have very little work to do at this point; run the installer, click a few times, and you’re done. Your application will be installed, runtimes put where they belong, and even the ODBC driver installed. But you’re not completely out of the woods yet.

Test connection to the server You’ve seen the scenes in heist movies where the characters have spent most of the film preparing to make a big getaway, and as they’re about to leave, the stolen money safely in the trunk, they each look at each other, and say, “I thought YOU had the keys!” Not far from that is the situation where you install the application at the customer’s site, start up the main program, and are confronted with an “Unable to connect to server” message. “I thought you already checked to make sure we could connect to the server,” you nicely ask your contact. “Well, we did. Something must have changed in your program.” And so it goes. So this step isn’t simply a reminder to test the connection from the client machine to the server before installation, but a recommendation to prepare for the inevitable failure after installation. I always provide a ‘test connection’ tool inside my applications that is available when a developer (or other qualified personnel) runs the app. Here’s how it works. First, provide a “Test connection” menu option under the Tools menu pad (or wherever you provide access to utilities and the like). This menu option is only available, however, when a user with ‘developer’ (or perhaps, ‘administrator’) credentials runs the app. This can be done via a test for the existence of a special file: if file("user_is_dev.txt") * enable and display "Test connection" menu endif


or just testing if the current user has ‘developer’ permissions. I like the former because it lets me run the system as a regular user but still test the connection independent of the rest of the system. The “Test connection” menu option then uses the connection string parameters currently in use to connect to the server, and if the connection attempt fails, the AERROR details are displayed right there in the dialog. This makes troubleshooting easier, as you don’t have to spelunk through the ZLOG error log table. This is one of those tools that should be in your application’s framework; write it once and use it forever. Now you’re done. Well, until the bug reports and enhancement requests start rolling in. But that’s another story.

Licensing A common question on the MySQL mailing lists has to do with licensing. As you know, there are two versions of MySQL; the Community version and the Enterprise version. However, traditionally, the MySQL site hasn’t made it clear when you can use the community version and when you were required to buy an Enterprise license. This is actually two questions – a technical one and a legal one. The technical answers have recently been addressed in a very descriptive page that helps differentiate the two. http://www.mysql.com/products/which-edition.html The legal issues, unfortunately, aren’t quite as clear. You can spend hours reading through the licenses and the GPL FAQ, and still come away with only a muddy understanding. This is one of those places where a few examples would be terrific help. The cynical among us might think this is done deliberately to push those who are uncertain into buying a license, ‘just to be safe’. Perhaps, but one oughtn’t ascribe to malice where ineptness could be to blame. Their lawyers just write the legal stuff; they don’t have to use it! How many of us write flawless user documentation? Nonetheless, you have a VFP application. You want to use MySQL. Do you need to buy a license? The short answer is that you can write a VFP app to connect to MySQL and not have to pay for a commercial license as long as the application isn’t resold. But there are some scenarios with potential grey areas, so let’s spell them out in more detail. Remember, I am not a lawyer, nor do I play one on TV, or even on the playground with the other boys and girls. I don’t even look like a lawyer. Let’s break these scenarios into two groups; those where you are an employee of a company and building apps for that company, and those where you are a consultant (either independent or as part of consulting group) and build custom apps for other companies.

Internal applications 1. You are employed at a company and build systems with VFP and MySQL for the company’s internal use. No license required. 2. You are employed at a company and use VFP and MySQL to build a website for use by the company’s customers – product catalogs, on-line store, knowledge base, search engine, and those sortsof things. No license required.

402


3. You are employed at a company and use VFP and MySQL to build a product your company resells. For example, you work at a software company that makes an email server that uses MySQL as the database to store data about users, accounts, and email messages. Your company’s business is selling and supporting that email server. Your company needs to pay a license fee for the inclusion of MySQL with your product. 4. You are employed at a company and use VFP to build a product that can connect to MySQL (and, perhaps, but not necessarily, other back-end databases). For example, you work at a software company that produces genealogy software. VFP provides the user interface and the business logic. When the user installs your product, they have to choose a database – either native VFP tables, MySQL, or another SQL database. Your company’s business is selling and supporting that product. No license required, because (1) the customer has to acquire and install MySQL themselves, (2) the use of your product isn’t dependent on MySQL, and (3) you don’t ship MySQL with your product. 5. You work at a company and use VFP and MySQL to build a product that your company gives away. For example, your company sells power generation equipment, like transformers, both through distributors (who sell to small customers) as well as directly to larger, more sophisticated end-users like power companies. The selection and configuration of the proper equipment can be a complex task, so your company has produced software that helps its customers do this in an automated fashion. The software by itself doesn’t do much, but it helps your sales force sell to their customers – both distributors and end-users. So your company gives it away to the distributors and customers in order to gain an edge. Some folks argue that a license is needed here, because you're distributing MySQL with your application. Others say that no license is required, since your company isn’t selling the product. If your company crosses the line where the configuration software is required to use the power generation equipment that your company sells, then, boom, yes, license required. What do I think? I think I can see both sides, and that if I was in this position myself, I'd shoot the MySQL folks an email.

Consultant Now, let’s look at the differences if you’re an independent contractor (or work at a consulting firm) and you build custom systems with VFP and MySQL for your customers. If you provide a copy of MySQL along with your application, it could conceivably be argued that you are reselling your application, and thus you owe a license fee. I don’t think this is plausible, but nonetheless, it probably makes sense for your customer to download and install MySQL, and you provide the VFP application that connects to it. You can, of course, use your application to configure and set permissions for the MySQL installation. With all of this said and done, step back and look at the big picture. The license for a single MySQL installation runs around $600. No client-access licenses needed, either, unlike some of the competition, who want you to pony up for every Tom, Dick, and Susy who wanders by a workstation. Don’t know about you, but my custom applications typically start at $50,000 and ratchet up from there pretty quickly. If a $600 license fee for the back-end database is going to sour the deal for a customer, then perhaps that customer should, uh, be working with someone else.

The GPL There is yet another twist in all of this because of the dual-licensing nature of MySQL both as an open source product as well as one released under a commercial license. If you resell your


application, but provide the source code under the GPL, you may not need to purchase a commercial license. At this point, even my interpretation gets fuzzy, so if this is an important issue to your situation, it’s time to visit the folks in the wood-paneled offices downtown.

What next? Experienced developers will tell you that the development and successful installation of an application takes 90% of your time, and the maintenance and upgrades take the other 90% of your time. Now that we’ve laid the groundwork for the first 90% covered, it’s time for you to start building your VFP/MySQL system. As you get started, you’ll undoubtedly have more questions. Where do you go for help? First, of course, is the online help ( don’t tell me you haven’t gone to the MySQL website and scanned through the documentation yet!), as well as the DuBois book. Both are indispensable references. Next are online lists. The official MySQL mailing list is found under “Community” on the MySQL website (www.mysql.com). There are other resources as well; I simply prefer mailing lists as opposed to forums. And the Pro* lists (profox for Visual FoxPro and prolinux for Linux) at leafe.com, while not as high in traffic, have a decidedly open source bent for VFP folks. Let me provide one more tip, particularly if you’re just getting started with MySQL, but envision a long development road ahead of you. Subscribe to the MySQL mailing lists you’re interested in, and store those messages away in a folder. Even if you stop MySQL development for a while, keep downloading those messages. Eventually you’ll accumulate a knowledge base of real-world MySQL information that can’t be beat. I have a folder with messages going back years, and it’s been very helpful to be able to search; many a time the question I have has already been asked, and answered, saving me a posting and the resultant wait.

Conclusion/Summary With the completion of this chapter, you now have all the pieces to start building client-server applications with Visual FoxPro and MySQL. There is obviously a lot more to MySQL than what we covered in these 22 chapters – transactions, replication, backup strategies, optimization, automated installations, and more. But as they say, shipping is a feature. Furthermore, as the title of this book indicates, this is about building apps, not about administration. Detailed coverage of these other topics can be found in many MySQL books, and they really don’t have that much to do with VFP, so now is a good time end this book. I’ll continue to write more, so look for whitepapers and articles at www.hentzenwerke.com. If you want to be notified when a new one is available, sign up on our mailing list. As I said at the beginning of this book, the combination of VFP and MySQL make a compelling weapon in your arsenal of application development tools. Not a day goes by when I fire up VFP and MySQL without a grin crossing my face. “Let’s see. What can I build today?” Have at it! Updates and corrections to this chapter can be found on Hentzenwerke’s Web site, www.hentzenwerke.com. Click “Catalog” and navigate to the page for this book.

404


Index

405

Index Note that you can download the PDF file for this book from www.hentzenwerke.com (see the section “How to download files” at the beginning of this book). The PDF is completely searchable and will provide additional keyword lookup capabilities not practical in an index.

Symbols

0000-00-00, 235 2207, Port, 111 32 Bit Compatibility, 13 3306, Port, 43, 66, 111, 133, 400 64 Bit Version, 13

ALLTRIM function, 229 Anonymous Account, 46 Anonymous user, 88 Anti-virus software, 49 anychange(), 342 append blank, 299 Architecture, 295 asc(), 307 ascii(), 307 asin(), 301 Assigning foreign keys, 200, 369 at_c(), 307 at(), 307 atan(), 301 atan2(), 301 atc(), 307 atcc(), 307 atcline(), 307 atline(), 307 Auto-increment, 232, 365 Autocomplete, 64

-A-

-B-

A table must have at least 1 column, 241 Abs(), 301 Access denied for user 'abc'@'localhost' (using password: YES), 76, 105 acos(), 301 Activate InnoDB, 146 Add New Connection button, 134 adddate(), 312 Adding a user, 86 Adding minor entities, 354 Additional connection string options, 109 addtime(), 312 Administer MySQL, 75 Administrator, 9 Advanced login options, 133 Advanced toolbar, 166 AERROR(), 237 alltrim, 307

Backup, 3 Backup facility, 398 Bad credentials, 160 Base Directory, 144 Basedir, 175 Batch files, 20 BDB, 145 Benchmark, 248 Berkeley Database, 145 Best practices, 181 between(), 303 Bigint, 187 BIN directory, 75 Binary, 189 Binary combination of flags, 111 Binary distribution, 56 Bit Arithmetic, 250 BIT field, 236, 249, 252

_cliptext, 109 -h flag, 90 -p flag, 89 .err file, 72,81 .ini files, 53 .msi file, 124 .pid file, 72, 81, 145 .sock file, 20, 72, 81 "I don't know", 259 "Don't Optimize", 106 /etc/init.d, 72 /usr/bin, 75 << and >> delimiters, 228, 339 - 0-9 -

406


BITAND function, 250 BITCLEAR functions, 251 BITOR function, 250 BITSET-like function, 257 BITTEST, 249, 251, 256 Bitwise functions, 250 Blank functions, 305 BLOB fields, 215258 Blobs, 189 Blocking port 3306, 50 Business logic, 379

-CCall, 382 Calling stored procedures, 387 Can't connect to MySQL server on '192.168.1.11' (10061), 105 Can't create table, 376 Cascade, 204 Cascading deletions, 371 Case studies, 9 Case-sensitivity, 64 case(), 303 Catastrophic errors, 239 cdow(), 311 ceiling(), 301 Certified Server, 9 Char, 188 char_length(), 308 charset(), 315 Child relationships, 332 Child tables, 333 chr(), 307 chrtran(), 307 chrtranc(), 307 Client Installation, 77 Client machines, 24 Client package installation, 65 Client tools, 20 Client-Server, 2 Client-server functions, 335 Close the connection, 107 Closing all connections, 108 Clustering, 3 cmonth(), 311 coalesce(), 304 collation(), 315

columns_priv tables, 179 Command Comparison, 296 Command contains unrecognized phrase/keyword, 246 Common migration errors, 240 Common SQLEXEC, 253 Community Edition, 9 Comparing, 189 Comparison, 303 concat_ws(), 307 concat(), 307 Config filename, 143 Configuration, 75 Configuration file, 119, 148 Configure Instance, 140 Configure Service, 141 Connect to MySQL server, 18, 93 Connection name, 135 Connection parameters tab, 134 Connection string, 108 Connection string options, 109 connection_id(), 314 Connectivity error, 388 Constraint actions, 200 Constraint name, 377 Conversion Issues, 295 Convert VFP dates, 232 Copy SQL to Clipboard, 227 cos(), 301 Could not find driver, 114 cpconvert(), 315 cpcurrent(), 315 cpdbf(), 315 Create connections, 331 Create Database command, 115 Create procedure, 381, 390 CREATE TABLE, 225 Create table engine = MYISAM, 194 Creating a database, 115, 191 Creating primary keys, 199 Creating stored procedures, 379 Creating tables, 192 Credentials, 76, 87 ctod(), 311 ctot(), 311 curdate(), 311 current_date(), 311

Index

current_time(), 313 current_timestamp(), 311 current_user(), 314 CursorAdapters, 18, 283 curtime(), 313 Custom Install, 95

-DData cleansing, 239 Data conversion, 219 Data Definition tasks, 167 Data Directory, 144 Data functions, 305 Data Manipulation Language, 117 Data Manipulation tasks, 169 Data Migration, 219 Data privileges, 173 Data Source Name, 99 Data too long for column, 241 Data types, 185 Database design, 181 database(), 305 Datadir, 175 Date functions, 310 date_add(), 312 date_format(), 312 date_sub(), 312 Date-time fields, 214 date(), 310 Date/Time types, 187 Datetime datatype, 187 datetime(), 311 day(), 311 dayname(), 311 dayofmonth(), 311 dayofweek(), 312 DB table, 179 DBC, 231, 247 dbc(), 305 dbused(), 305 Debug Output window, 237 DEBUGOUT command, 237 Decimal, 186 Default Schema text box, 140 Default Schema text box, 157 Default storage, 144 Default-storage-engine, 174

407

degrees(), 303 Delete, 299 delete() method, 347 deletebiz(), 355 Deleting minor entities, 354 Delimiters, 226, 380 Denormalized table issues, 346 Deploying Visual FoxPro Solutions, 393 Deployment, 393 describe command, 209 Determining the last value of an autoincremented field, 366 developer-only text boxes, 345 Development configuration, 17 Diagnostics panel, 104 difference(), 315 Direct Injection, 265 Disabled Temp directory, 144 DispWarnings, 116 distributions, 56 DML SQL commands, 117 dmy(), 312 Double Precision, 186 dow(), 312 Downloading the MySQL ODBC driver, 93 Drop database, 192 Drop procedure, 386 Drop table, 229 DSN, 99, 106 DSNs vs connection strings, 114 dtoc(), 312 dtor(), 301 dtos(), 312 dtot(), 312 Duplicate entry, 241

-EEdit procedure, 386 elt(), 304 Empty dates, 235, 362 Empty functions, 305 Empty(), 305 Enable Strict Mode, 43 Engine defects, 266 Enum, 189 ENUM field types, 249, 261 enumerated, 261

408


Error 1292, 384 Error 1318, 384 Error 350 - Field must be a memo field, 106 Error files, 145 Error trapping, 236 Error: 'A table must have at least one column', 241 Error: 'Access denied for user 'abc'@'localhost' (using password: YES)' Error: 'Can't connect to MySQL server on ...', 105 Error: 'General failure', 104 Error: 'Incorrect number of arguments for PROCEDURE', 384 Error: 'Permission denied', 81 Error: 'SQL_ERROR', 104 Error: 'Unknown MySQL server host', 105 Error: Host '192...' is not allowed to connect to this MySQL server, 10 Escape string function, 234 Escaping the character, 234 evaluate() function, 245 evl(), 305 exit, 84 Exit interactive environment, 84 exp(), 302

-FFailure to trap data type, 266 Failure to validate data, 265 fdate(), 305 Fedora Core, 55 Field Attributes, 195 Field must be a memo field error, 106 Field name lengths, 231 Field validation, 246 field(), 304 File DSN, 100 File size, 259 filetostr(), 307 Filter parameter, 317 find_in_set(), 304 Firewall conflicts, 66 Float, 186 flock(), 305 floor(), 302 Foreign key constraints, 371

Foreign keys, 184, 372 Format SQL commands, 227 found_row(), 306 FoxPro 1.0, 287 from_days(), 313 ftime(), 305 Full outer joins, 363 Func and proc tables, 179 Function Cross Reference, 296, 299 Functions, 379

-GGeneral failure message, 104 get_lock(), 305 getautoincvalue(), 306 getresults(), 341 Global application object, 396 Global disconnect option, 108 gomonth(), 312 GPL, 402 GRANT command, 147 greatest(), 304 GUI tools, 20

-HHard-coded values, 226 Hashed version, 89 Heap, 145 Host '192.168.1.7' is not allowed to connect to this MySQL server, 10 Host table, 179 Host value, 87 Hostname declaration, 88, 136 hour(), 312 hwctrl.vcx, 319 hwlib.vcx class library, 335

-Iicase(), 303 icase(), 308 id(), 314 if(), 304 ifnull(), 305 iif(), 304 Importing, 219 Improperly trapped delimiters, 264 in(), 304

Index

Include BIN directory in Windows PATH, 75 Incorrect number of arguments for PROCEDURE, 382 Incremental backups, 258 Indexes, 198 Information browser, 166 Infrastructure, 258 init.d, 72 inlist(), 304 InnoDB, 72, 145, 173, 176, 247, 376, 399 InnoDB parameters, 145 InnoDB storage engine, 200 innodb_data_home_dir, 175 Input parameters, 385 INS database, 221 insert(), 307, 309 Install client package, 65 Install MySQL, 31 Installation layouts, 69 Installation of the ODBC driver, 93 Installation package, 394 Installing MySQL on Linux, 55 Installing the MyODBC driver, 94 Instance Manager, 175 instr(), 307 int(), 302 Integer, 186 Interactive environment, 151 Interactive environment monitor, 83 Interactive MySQL functions, 300 interval(), 303 isalpha(), 308 isblank(), 305 islower(), 308 isnull(), 305 isupper(), 308

-KKey-value pairs, 109 Keys, 183

-Llast_day(), 312 last_insert_id(), 306 Launch MySQL server automatically checklist, 142

Leading spaces, 214 least(), 304 left(), 308 leftc(), 308 len(), 308 lenc(), 308 Licensing, 401 like(), 315 likec(), 315 Line continuation character, 85, 110 Line termination character, 227 Linux root user, 77 List structure, 206 Lists, 303 Literal, 234 ln(), 302 LOAD, 211 Load Data Infile, 212, 219 load_file(), 307 Localhost, 89, 111, 133, 136 Locate, 299 locate(), 307 Log files, 145 log(), 302 logic functions, 303 Long strings, 246 lower(), 308 lpad(), 308 ltrim(), 308 lupdate(), 306

-Mmakedate(), 310 maketime(), 310 Math Functions, 301 Max Connections parameter, 146 max(), 304 MD5 Checksums, 62 MD5 text string, 28 mdy(), 312 Mediumint, 187 Metadata, 173 Metadata location, 176 microsecond(), 313 Microsoft Notepad, 212 mid(), 309 Migration, 219

409

410


Migration toolkit, 211 min(), 304 Minor entities, 333 Minor entity relationships, 332 minute(), 312 Miscellaneous functions, 315 Mistype a driver name, 113 mod(), 302 month(), 312 monthname(), 311 Multiple connection strings, 396 Multiple data types, 234 my.cnf, 121, 148, 173 my.ini, 53, 119, 144, 148, 173 MyISAM, 72, 145, 173, 176, 247, 376, 399 MyISAM and InnoDB files, 176 MyISAM parameters, 145 MyISAM tables, 149 MyODBC, 93 myodbc3.dll, 99 myodbc3.lib, 99 Myslqdump, 397 MySQL Administrator, 119 MySQL Administrator download, 122 MySQL config file, 381 mysql daemon, 83 MySQL Front, 171 MySQL Maestro, 171 MySQL Monitor, 212 MySQL root user, 66, 77 MySQL scripts, 76 MySQL server, 25 MySQL service, 30, 53, 120, 142 MySQL user account, 77 MySQL variables, 78 mysql_install_db script, 90 mysql-administrator, 130 mysql.exe, 19 mysql.proc, 380 mysql.user table, 91 mysql> prompt, 84 mysqladmin, 77, 80, 88 Mysqld daemon, 19, 56 mysqld_safe, 80 mysqld-nt.exe, 49, 79 mysqld.exe, 25, 56, 79 mysqlhotcopy, 399

Mysqlshow, 82, 206

-NNaming Conventions, 182 Navicat, 170 net start mysql, 79 Net stop mysql, 30, 79 New accounts, 86 No Action, 204 No database selected, 240 noshow option, 227 notastar.txt, 212 Notepad++, 212 now(), 311 Null dates, 363 NULL functions, 305 nullif(), 305 NULLs, 259 Numeric types, 186 Nvl(), 305

-OObject browser, 164 Occurs(), 308 ODBC, 13, 17 ODBC Data Source Administrator, 98 ODBC Driver, 19 ODBC problems, 238 Old-style xBase, 295 On delete actions, 204 On Update actions, 204 Option=3, 111 Options, connection string, 109 ord(), 307 Output parameters, 385 Overloading tables, 184

-PPad CHAR field to full length option, 107 padc(), 308 padl(), 308 padr(), 308 Parameter list, 382 Parameter separator, 110 Parameterized queries, 267 Pass connection parameters, 326

Index

Passing parameter to a stored procedure, 389 Password concepts, 87 Path to executable, 174 Performance issues, 247 period_add(), 312 period_diff(), 312 Permanent backups, 258 Permission denied error, 81 PhpMyAdmin, 171 PID file, 20 Ping button, 159 Ping the box, 105 Pl(), 302 Port 3306, 43, 66, 111, 133, 400 Port 3306, blocking, 50 Port 3307, 111 position(), 307 power(), 303 Primary keys, 183, 214 Privilege data, 177 Procedures, 379 process ID, 72, 145 Process listing, 73 procs_priv table, 179 Production data, 397 Production machine, 23 Production server, 393 Programming errors, 239 Proper(), 308 ps -aux, 80 ps command, 73

-QQuarter(), 313 Query Browser, 9, 139, 151, 161, 167, 212, 380 Query toolbar, 161 quit, 84 quote(), 307, 309 Quoted injection, 266

-Rradians(), 301 rand(), 303 rat(), 308 ratc(), 308

ratline(), 309 Real, 186 reccount(), 306 recno(), 306 Record validation, 246 Referential integrity, 4 Relational database theory, 181 Relational integrity, 246, 369 Relational rules, 200 Release_lock(), 306 Remote machine, 22 Remote views, 18, 247, 271 Reorganizing the migration code, 243 repeat(), 309 Replace command, 299 replace(), 307, 309 replicate(), 309 Replication, 3 Restrict, 204 Restrict access, 87 Restricted deletions, 371 Result area, 162 Result set, 117, 317 Retrieving empty dates, 362 Retrieving null dates, 363 RI setup errors, 375 right(), 309 rightc(), 309 rlock(), 306 Root access, 46 Root user account, 77 root user, Linux, 77 root user, MySQL, 66, 77 round(), 303 routine_schema field, 387 row_count(), 306 rpad(), 308 rpad(), 308 RPM installation scripts, 71 RPMs, 56 rtod(), 303 rtrim(), 309 Run the procedure, 382

-SScalability, 4 Schema, 136

411

412


Schema Privileges, 147 Schemata tab, 380 Scripts, 20 sec(), 313 second(), 313 seconds(), 313 Security, 3 Security Settings, 46 Security vulnerability, 263 Seek, 299 Selecting number of records, 365 Semicolon, 380 Server, 78 Server machine, 24 Server process ID, 72 Service, 78 Service removal tool, 51 service, MySQL, 30 Services applet, 120 Set, 189 Set a password, 77 SET CENTURY ON, 232 SET DATABASE TO, 116 SET HOURS TO 24, 232 Set Null, 204 Set passwords, 86, 88, 89 Set relation to, 299 Show databases, 207 show tables, 208 Show variables, 380 shutdown parameter, 85 Sidebar, 164 simplenav_withchild_andfilter1/2.scx, 322 simplenav_withchild.scx, 321 simplenav_withmysql1/2.scx, 325 simplenav.scx, 320 sin(), 303 skip-innodb, 146 Smallint, 186 Socket file, 20 Sort street number field, 362 soundex(), 315 Source distribution, 56 space(), 309 Special characters, 234 SPT, 115 SQL Injection, 263

SQL Injection example, 264 SQL Injection solutions, 266 SQL Pass-through, 18, 115, 219, 285 SQL_ERROR, 104 sqlconnect(), 107 sqldisconnect(), 107, 118, 246 sqlexec(), 115, 224, 343 SQLEXEC() wrapper, 246 sqlgetprop(), 109, 115 Sqlstringconnect(), 109 SQLyog, 171 sqrt(), 303 Starting and stopping as a service, 79 Starting and stopping in the DOS box, 79 Starting the server, 78 Startup entries, 72 Startup Errors, 159 Startup Variables, 143 Stop Service, 141 Stopping and starting the server- Linux, 80 Stopping the server, 78 Storage engines, 40 Store connection handle, 326 Store files, 258 Stored Connection, 133, 135 Stored Procedure (script) tasks, 170 Stored procedure location, 387 Stored procedures, 3, 246, 379 str_to_date(), 311 str(), 309 Strategic use of BIT fields, 249 strconv(), 309 strextract(), 309 String types, 188 StrToFile(), 109, 309 strtran(), 309 Structure of an existing MySQL table, 227 stuff(), 309 stuffc(), 309 subdate(), 312 substr(), 309 substring_index(), 309 substring(), 309 subtime(), 312 SuSE Linux, 55 sys(1), 313 sys(10), 313

Index

sys(11), 313 sys(2), 313 sysdate(), 311 System DSN, 100, 103 System Information functions, 314 System Tray Monitor, 131, 156 System variable, 78

-TTable structure, 227 tables_priv table, 179 tabletoform(), 342 tabletoform(), 353 tan(), 303 Task Manager, 80 Test connection, 400 Test data set, 395 Test For Empty Date, 235 Text, 189 TEXT fields, 215 text to ...textmerge, 339 TEXT/ENDTEXT commands, 227 Textmerge delimiters, 228 textmerge option, 228 Third party conversion tool, 211 Tick marks, 227 Time stamps, 184, 188, 205 time_format(), 312 time_to_sec(), 313 Time(), 313 Timestamp, 187 Tinyint, 187 tloc(), 313 tlod(), 313 to_days(), 313 Transactions, 247 Triggers, 246 trim(), 307, 308, 309, 310 TRUNCATE command, 241 truncate(), 302 Type, 136 Type the data, 211

ui_delete.scx, 345 ui_edit.scx, 339 Uninstall a previous version of MySQL ODBC driver, 93 Uninstalling MySQL, 30, 62 Unknown MySQL server host '192.168.1.777' (11001), 105 Unknown table, 241 Updating the password, 88 upper(), 310 Use, 299 User account, 70, 76 user account, MySQL, 77 User Administration node, 147 User DSN, 100 User Interface, 317 User rights, 148 User table, 178 user_info table, 178 User-defined function, 234 user(), 314 Username, 87 Using foreign key constraints, 374 uuid(), 314

-Vvalidate_data(), 338 Varbinary, 189 Varchar, 188 variables parameter, 78 Verify, 62 Version(), 314 VFP logicals, 249

-Wweek(), 313 weekday(), 312 weekofyear(), 313 Windows DOS box, 75 WinMd5Sum, 30 Work-a-day user, adding, 90

-X-Uucase(), 310 ui_add.scx, 336 ui_allinone.scx, 350

Xbase, 295 Xbase commands, 299 Xcase, 171, 211

413

MySQL Client Server

Recommend Documents