AUTOMATING WITH NODE.JS By Shaun Michael Stone © First Edition UK - 2018
AUTOMATIN AUTOMATING G WITH NODE.JS 1. AUTOMATING WITH NODE.JS 2. Introduction 1. Preface 2. End Goal 3. Structure 1. Part 1 2. Part 2 3. Boo Book k Coding Style 4. Code snippets 5. Prere Prerequisites quisites 6. Assumptions 7. Suggest uggestions ions 3. Technical Overvie Overview w 1. Technic echnical al Terms 1. CLI 2. Bash 3. Node.js 4. npm 5. npm dependency 6. Node Modules 7. ES6 8. Chrome V8 9. ESLint 10. JavaScript Promise 4. Part 1 - Examples
5. 001 - Arguments 1. Comparison 1. Bash Example 2. Node Example 2. Summary 6. 002 - Process 1. Summary 7. 003 – Query String 1. Summary 8. 004 – URL 1. Summary 9. 005 - OS 1. Summary 10. 006 - Open 1. Summary 11. 007 - File System 1. Summary 12. 008 - Zip 1. Summary 13. 009 - Colours 1. Summary 14. 010 - Question 1. Summary 15. 011 – Cloning 1. Summary 16. 012 – Branching 1. Summary 17. 013 – Retain State
1. Summary 18. 014 – Choose Template Template 1. Summary 19. 015 - Email 1. Summary 20. 016 - SMS 1. Summary 21. 017 - CSV 1. Streams 2. Summary 22. 018 - Shorten URL 1. Link Shorteners 1. Long URL 2. Short URL 2. Summary 23. 019 - Minimist 1. Summary 24. 020 - Build 1. Summary 25. Part 2 - Build Tool Tool 26. Scenario 1. Development Meeting 1. Required Tech 2. Required Repositories 27. Build Tool Planning 1. repositories 2. src 1. commands
2. constants 3. creators 4. helpers 5. setup 6. nobot.js 7. Games List 28. Commander 29. Configuration 30. Constants 31. Helpers 32. Setup 33. Command - Setup 34. Command - Template 35. Command - Game 36. Creator - Rock Paper Scissors 37. End to end 38. Wrap up
AUTOMATING WITH NODE.JS All rights reserved. Every precaution was taken in preparation for this book. boo k. However, the author assumes no responsibility for errors or omissions, or for damages that may result from the use of information. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form, or by any means, electronic, mechanical, photocopying, recording, or otherwise without the consent cons ent of the author.
Introduction “I will always choose a lazy person to do a difficult job because a lazy person will find an easy way to do it.” – Anonymous
Preface Being in the technical field presents itself with some fun… and some not so fun tasks, we can all agree upon that. For the not so fun bits I try to filter out all of the repetitive administrative work with something that I’m better known for as a developer; writing code. Yes, I can be a bit lazy, lazy, but that’s because I value my time. You’d You’d be surprised at how much time goes into updating comments on Jira, zipping files and emailing them to colleagues, updating configuration files, and copying and pasting directories. Arghh, I need to stop listing these before I fall asleep. At a previous job I found myself doing things that could’ve easily been automated by a script. Anything that feels repetitive should ring alarm bells for you. If it doesn’t, then you should change your mindset now. now. Look at what you do on a daily basis, think about it, read this book, and see if it changes your perspective. Ideally it should. On one weekend, after noticing repetitive tasks at work, I took note of the steps involved in these tasks from start to finish, and set out to build a suite of commands that would automate them. It proved to be efficient for both me and some of my teammates, and it gave me more time to concentrate on Reddit… I mean, my work. I remember reading a funny story about a programmer who automated anything that took longer than ninety seconds. His coffee machine was connected to the local network, and he sent commands to it and timed how long it took for him to walk over to pick up his freshly brewed cup. He even programmed it to send a text message via Twilio Twilio to his wife if his machine was logged in fifteen minutes after the end of the working day, saying he was working late that night.
Being fairly accustomed to using Bash scripting in the past on a Linux virtual machine, I decided initially it was the right tool for what I wanted to achieve. I’d need access to the file system. I could make use of the powerful searching commands, store variables, read in standard input from the user, and use conditional statements to decide on how to proceed. Perfect! But then I thought, I wonder if I can achieve the same with Node JS? I created the bash version initially, initially, but digging further, I learned I could create the project with npm directly. So I rewrote the project and pr esented it to the team. The great news was that me and my team were allocated time to work on the project during work hours, and one of the technical architects was keen to integrate this tool into our workflow. workflow. Winning! There are two ways we can implement the code you will be learning in this book. You You can treat it as a global list of commands that behave in the same way as an alias on a terminal, or you can create a build tool that deploys your project, taking care of all the tedious tasks you are used to doing. This book will help you build something along the lines of what I have, but it’s obvious to point out that every company’s company’s workflow follows a different d ifferent path and set of business rules. Don’t worry though, section two of this book explains a good way of identifying and detailing your workflow. workflow. Once you have identified this path and the associated ass ociated workflow, workflow, it should be pretty straightforward s traightforward to apply the knowledge acquired from this book.
End Goal Let’s Let’s not beat around the bush. Once you’ve finished reading this book, you should be able to create global commands and a working bespoke Node build tool that allows you to automate the repetitive tasks you hate doing. This build tool will be shaped around your company’s goals, or your own. Either way, the intention is to make your life easier. easier. Because life is hard h ard enough as it is, right?
Structure The book is structured structured into two parts:
Part 1 The first part is a collection of recipes, or building blocks that behave as individual global commands. These can be used as you go about your day, and can be called at any time to speed up your workflow or just for pure convenience. It begins with simple examples example s so you can get to know more mor e about Node’s standard library, then moves into more practical implementations. Each recipe corresponds with the ‘examples’ directory found in the repository. All of the examples can be found here: https://github.com/smks/nobot-examples
Part 2 The second part is a walkthrough of creating a cross-platform build tool from the ground up. Each script that achieves a certain task will be its own command, with a main umbrella command – usually the name of your project – encapsulating them all. Instead of using Gulp or Grunt, we will be using npm directly. directly. The plan is to keep this tool as lightweight as possible. I will be calling the project Nobot, because I love naming na ming projects, it’s it’s an obsession. The implementation can be found here: https://github.com/smks/nobot
Above shows a high level overview of the repositories we will make use of in part 1 and part 2 of this book.
Book Coding Style This book uses examples when working on a Mac and sometimes Windows. You You may occasionally see different output. Some of the code examples may wrap onto the next line due to spacing limitations. The coding style follows AirBnb coding standards with ESLint. A few rules have been overridden.
Code snippets The book will have a lot of code snippets, as you’d expect.
Below is how I would demonstrate a code example. It begins with the name of the script, followed by code snippets, and sections of content in between to explain what is happening.
my-script.js
This is where I introduce you to what on earth is going on. // start of script
console. console .log log( ('this is part 1 of my-script.js' my-script.js') );
Above is the first bit of code. This is where I bore you of the details of what’s going on, or what will happen next. console. console .log log( ('this is part 2 of my-script.js' my-script.js') ); // end of script
Below is the output of the script ‘ my-script.js’ $ node my-script.js this is part 1 of my-script.js this is part 2 of my-script.js
Did you know
When I’m feeling a bit generous, I provide some explanations to relevant areas associated with the code that we write.
Immutability in the context of programming - an immutable object is an object whose state cannot be changed once created. This can be useful because when you pass references of that object around, you can be rest assured other procedures will not be cheeky and modify it. Coding time
When you see this pencil icon, get ready, because it’s it’s time to roll up your sleeves and get coding!
Running a terminal command
When I need to use the CLI, it may show as a single line. node index.js
For multi-line, I will prefix the first line with a dollar sign. $ npm install fs-extra fetching fs-extra...
Prerequisites 1. A Laptop or Desktop. 2. Internet access. 3. A GitHub account with with SSH set up correctly. correctly. 4. Ensure you are using the latest version of git git to avoid legacy issues. 5. Make sure you have Node installed. This can be downloaded here for your Mac or Windows machine: https://nodejs.org/en https://nodejs.org/en.. This book uses a minimum version of: 6.9.1. At the time of writing, it should be fine to use any version above this. 6. Motivation. Please stick with with it. The time you invest now will pay off off in the long run.
Assumptions It’s It’s assumed you have a simple understanding of JavaScript and GitHub. A basic idea of the CLI, and minimal - or no - experience of Node JS. All third party implementations are correct at the time of writing. Node throughout the book may be referenced as: Node, Node JS or Node.js but all references refer to the same technology.
Suggestions Please feel free to suggest or contribute on GitHub (Raise a pull request) to the code examples as you see fit, or any possible typos in this book. You You can also contact me via any of the social networks.
GitHub - https://github.com/smks Twitter - https://twitter.com/shaunmstone Facebook - https://www.facebook.com/automatingwithnodejs YouTube - http://www.youtube.com/c/OpenCanvas Or connect with me on LinkedIn for business-related requests. LinkedIn - https://www.linkedin.com/in/shaunmstone
Technical Overview Just to make sure we’re all on the same page, here are some of the terms in this book that you should understand before proceeding. Feel free to skip past them if they’re already familiar to you.
Technical Terms CLI Command Line Interface - is a textual interface for interacting with your computer. computer. It is essentially a text-based application which takes in text input, processes it, and returns an output. When interacting with the examples in this book, you will need to open up a CLI and type in commands to make things happen, rather than clicking buttons and tabs with a mouse. If you are on Windows, this will be the Command Prompt (CMD) or PowerShell. If on Mac or Unix like systems, it will be the Terminal.
Bash Bash is a shell command c ommand processor that runs in a CLI. You You can write Bash scripts, and run them to execute a sequence of commands. You You might first clone a repository, create a branch, add a text file with content, stage the file, commit it, and then push back to the remote repository all in one go. This would mean you wouldn’t have to type out each command separately and is handy for automation. The reason this book does not use Bash is because – at the time of this writing – Windows does not fully support it, and we want our project to be cross platform. So we will be writing JavaScript with Node so our scripts will run on Windows as well. Here is an example of a Bash script.
new-branch.sh #!/bin/bash # 0.0.1
git checkout master git pull origin master git checkout -b $1
Node.js When you open up the CLI and type
node
, you are interacting with the the node
executable installed on your machine. When you pass a JavaScript file to it, the node executable executes the file. Node is an Event-driven I/O server-side JavaScript environment based on Google’s Google’s V8 engine. It was designed to build scalable network applications. It processes incoming requests in a loop, known as the Event Loop, and operates on a single thread, using non-blocking I/O calls. This allows it to support a high volume of concurrent connections. Node has two versions available on their website to download: LTS
It stands for Long Term Support, and is the version of Node offering support and maintenance for at least 18 months. If you have a complex Node app and want stability, stability, this would be the choice for you. Support and maintenance is correct at the time of writing. Stable
Will have support for approximately 8 months, with more u p-to-date features that are released more often. Use this version if you don’t mind having to keep updating your application so you can keep in line with ‘on ‘ on the edge’ technology. technology. I have opted to use the LTS LTS version so that companies who are tied down with their version of Node will more likely be able to run the code examples and implement the build tool demonstrated in this book.
npm
When you download Node, it optionally gets bundled with a package manager called npm. It stands for Node Package Manager, and is the de facto for managing your external dependencies. If you wanted to use a library such as React or Angular, all you need to do is run
npm install [package name]
download/install the package into your project’s
node_modules
, npm would then
directory, directo ry, so it’s
ready to be used in your app. But this is not the only thing npm does after running this command. It also adds a record of this package to your project’s dependencies list in your
package.json
.
This is very handy, as it means that your project pr oject keeps track of all its dependencies. But it gets much better.
Please note: As of npm 5.0.0, installed modules are added as a dependency to your
package.json
option
--save
file by default. Before this this version, you would have to add the the
to do this.
Any developer wanting to use your app (including yourself from another machine) can install all dependencies with just one command:
npm install
. When
running this command, npm goes through your dependency list in your project’s package.json
file, and downloads them one by one into the
node_modules
directory.
To be able to use this dependency management goodness goo dness in a freshly fresh ly created project, all you need to do is run
npm init
series of questions, and create an initial
. This command will take take you through a package.json
file for you. This file, aside
from keeping track of your project’s dependencies, also has other information about your project, such as: project name, project version, repository details, author name, license, etc.
npm dependency {organisation}/{package} # examples
facebook/react apache/cordova-cli expressjs/express
Each dependency in the npm ecosystem has to have a unique identifier on the public registry, registry, otherwise this would cause conflicts. c onflicts. Think of it like checking into a hotel, if you wanted room number seven because be cause it’s lucky, lucky, but someone else is already in there eating bread and olives, it means you’ll have to settle for a different room. Same applies to package names. Anyone can create their own package and publish it to the registry, registry, just make sure the package name you decide to use is available. When I try to install the ‘express’ package, it will use the one created by the Express organisation. I can’t publish a package called ‘express’ anymore as this is already taken.
Node Modules When we want to break bits of code into separate files, we treat them as ‘modules’. These modules can be imported into other modules. In this example, I want to use code from the file
b.js
in my current file called
in the same directory for the following example.
a.js const b = require require( ('./b.js' './b.js') );
console. console .log log( ('From a.js: running code in the file b.js' b.js') ); b() (); ;
So above, we are importing the code from the file below:
a.js
. Both files sit
b.js const arsenalFanChan arsenalFanChant t = () => {
console. console .log log( ('We love you Arsenal, we do!') do!'); } module. module .exports = arsenalFanChan arsenalFanChant t;
Above shows
module.exports
. Whatever is assigned to this object from your
JavaScript file, can be retrieved by doing a Now, Now, when we run script
a.js
require
from another JavaScript file.
.
$ node a.js From a.js: running code in the file b.js We love you Arsenal, we do!
The
require
function does the following:
1. It finds the path to the the file you pass in. 2. It determines the type of the file (JavaScript/JSON etc.). 3. Applies wrapping of the the code to give file private scope (variables inside this file are limited to the file only making the variables private, unless exported). 4. Evaluates the loaded code. 5. Caches the import so that we don’t have to repeat these steps when requiring the same file somewhere else. The way
require
resolves a path is very smart:
If you specified a relative or absolute path, it will load the module from that path. You You don’t even have to write the module’s file extension, as the require method will add it for you. You You can even write a directory name, and it will look for a file named
index.js
in that directory. directo ry.
If you just passed a module name without a path, the
require
method will
use a searching algorithm to find your module. It will look through its core modules, then traverse up the directory tree looking through node_modules directories, and finally, if it still hasn’t found your module, it will look for it in the directories specified in its directory paths array. If we try and require a file, but it does not have anything exported, then its value will be
undefined
. To expose the function
arsenalFanChant
we assign it to
module.exports
.
If you’ve used a language like Java, you would have come across a similar idea of importing packages. These modules are used to avoid scope conflicts and break our code up so it’s more maintainable. Ain’t nobody got time for thousands of lines of code in one file!
ES6 ECMAScript 6 or ECMAScript 2015 is a significant update to the JavaScript language, incorporating many useful features and syntactic sugar. We We will be using some of these features in the scripts we write. A list of these features and examples can be found on this website. http://es6-features.org http://es6-features.org.. Node JS supports most of the new standard (not all) at the time of writing, so there won’t be a need to transpile our JavaScript with Babel in this book. // examples of syntactic sugar from ES6
class Homer { static speak speak() () {
console. console .log log( ("doh!" "doh!") ); } } const obj = { x x, , y }; const odds = evens evens. .map map(v (v => => v v + 1);
Chrome V8
Chrome V8 is a JavaScript engine developed by Google, which is used by the Chrome browser and Node.js (amongst other applications). Powered by C++, it compiles JavaScript to native machine code (supporting many CPU architectures), and then executes it. This means that it benefits from a much faster execution compared with traditional techniques such as real-time interpretation. V8 also handles memory allocation for objects, and if an object in memory is no longer needed, an operation known as garbage collection is applied to remove it.
ESLint The JavaScript we write should follow a certain standard: Spacing between
if
statements and blocks, indentation, variables and f unctions should all remain consistent. It’s something I think is very important as it alleviates my OCD. In this book we’ll be following AirBnb coding standards with a few overrides declared in a
.eslintrc
file. ESLint will flag up anything that that doesn’t abide by
these standards. This will be installed as a dependency via npm. The file below will be incorporated into our projects.
.eslintrc { "extends": "airbnb" "extends": "airbnb", , "rules": "rules" : { "no-console": "no-console" : 0, "linebreak-style": "linebreak-style" : ["error" "error", , "unix" "unix"] ], "no-use-before-define": "no-use-before-define" : ["off" "off"] ], "comma-dangle": "comma-dangle" : ["error" "error", , "never" "never"] ], "global-require": "global-require" : ["off" "off"] ], "import/no-dynamic-require": ["off" "off"] ] }, "env": "env" : { "browser": "browser" : false, "node": "node" : true } }
In our projects, we can run the following command to check the rules are being followed. npm run lint
JavaScript Promise You pass your friend ten pounds and say, say, ‘Hey buddy! Can you get me a pizza?’. Because he is such a good friend, he says, ‘Yes. ‘Yes. I promise.’ function friendGetsPizza friendGetsPizza() () { return new Promise((resolve Promise((resolve, , reject) => { // ... do things to get pizza // Scenario 1 - He got the pizza
resolve( resolve ({ name name: : 'Margherita' }); // Scenario 2 - He did not get the pizza
reject( reject ('cannot be trusted') trusted'); }); }
Then, when he has successfully got it, I want to eat it. friendGetsPizza() friendGetsPizza() .then then((pizza) ((pizza) => { console. console .log log( ('now eating pizza', pizza', pizza pizza. .name name) ); });
But… what if he is a terrible friend and doesn’t come back, and eats the pizza himself? He did NOT fulfil his promise. friendGetsPizza() friendGetsPizza() .then then((pizza) ((pizza) => { console. console .log log( ('now eating pizza', pizza', pizza pizza. .name name) ); }) .catch catch((e) ((e) => { console. console .log log( ('take friend out of life and move on because he', he', e) e); ; });
In this case, the
catch
function will will be called rather than the
because the promise was rejected.
then
function,
Part 1 - Examples In GitHub, make sure you have created an account if you want to write these scripts from scratch. Alternatively, you can browse the completed scripts sitting on the
master
branch of the repository mentioned below. below.
1. Fork the repository under your own name. This can be done by clicking clicking the Fork button on the GitHub page here https://github.com/smks/nobotexamples.. So rather than the repository being under my own name ‘smks’ it examples will be under yours instead. 2. Clone your forked repository to your own machine using git clone [your-forked-r [your-forked-repository-url] epository-url]
3. Change into the root of the cd nobot-examples nobot-examples
npm install
repository you’ve just cloned.
.
4. Switch to branch 5. Run
nobot-examples
develop
by running the command
git checkout develop
.
.
6. Follow along with following following examples 001-020 by writing code. 7. Happy coding!
Please note: Whenever you see a file called copy of it and rename to examples by running
config.json
npm run setup
config.example.json
you need to make a
. This can be done automatically for all
in the root of the repository. repository.
001 - Arguments Because we will be interacting with the CLI, there is the requirement to pass our own input. With Bash, arguments are passed by number and prefixed with a dollar sign.
Please note: There is no need to write out the Bash examples, they are used for demonstration only. only.
Comparison Bash Example Before we try this out in Node, let’s see how this would look in Bash. In this example, we are running the Bash script and passing a name to it as an argument. Since this is the first argument, we can access it in the script using
$1
.
$ bash my-bash-scrip my-bash-script.sh t.sh 'Fred Flintstone' ... running script
my-bash-script.sh name="$1" # name is now - 'Fred Flintstone'
Node Example We can’t do this with Node as conveniently. conveniently. Instead we can use a native object called
process
, which includes – as the name would imply – all the values related
to the process of the script. For the time being we just want to obtain the arguments. Let’s see what happens when we do this using Node. Suppose we have a script named
my-node-script.js
node my-node-script.js 'Fred Flintstone'
When running the above, we are initiating a new process. So what is in this process
object? An array called
argv
.
my-node-script.js const args = process process. .argv argv; ; /* args array is now [ '/Users/shaun/.nvm/versions/node/v8.7.0/bin/node', '/Users/shaun/Workspace/nobot '/Users/shaun /Workspace/nobot-examples/examp -examples/examples/001/hello.j les/001/hello.js', s', 'Shaun' ] */
As you can see above, we have three elements in the array. The first one is the full path of the node executable, the second one is the full path of the script we are executing, and the third one is the first argument we passed from the CLI. It’s It’s a common misconception to think the first f irst argument you pass ‘Fred Flintstone’ is referenced as the first element of the array. array. So remember that your arguments will start from the third index of the process array onwards. Usually there is no need for the first f irst two elements of the process proces s array, so let’s let’s remove them. const args = process process. .argv argv. .slice slice( (2); /* args is now [ 'Fred Flintstone' ] */
Now that we have removed the array elements we don’t need, we are left with the arguments we passed to the script.
Now it is time to code. Write out the following.
examples/001/hello.js const args = process process. .argv argv. .slice slice( (2); const [name] = args args; ; if (name === undefined) {
console.error console. error( ('Please pass a name, e.g. node hello.js Shaun') Shaun'); process. process .exit exit( (0); } console. console .log log( (`Good day to you, ${name ${name}` }`) );
As discussed earlier, we are removing two elements that we don’t need from the process array. Using some ES6 syntactic sugar, we can grab the first element and assign it to a constant called
name
. This is the same as doing this: this:
const name = args[ args[0 0];
The
if
statement is to make sure the user has actually passed an argument argument to
our script. If it’s undefined it means the user called the script without passing any arguments. If this is the case, we want to exit the script by calling exit on the This means that it won’t reach the final console log saying
process
object.
Good day to you, ${name}
Instead, it will print the following and then terminate the script. Please pass a name, e.g. node hello.js 'Shaun'
Now let’s see what happens when the user runs our script and passes an argument. $ node examples/001/ examples/001/hello.js hello.js 'Shaun' Good day to you, Shaun
.
Summary We created our first Node script, which takes the user’s name as an argument a rgument and prints it to the CLI. The reason we can do
node filename.js
is because once we’ve installed Node, our
operating system will have an executable identified as node, stored globally so you can call it from any location in the CLI. The JavaScript file
hello.js
, which
contains our script, gets passed to the Node executable along with the string ‘Shaun’. The string is treated as an argument, and it will be passed to the process.argv
array. array. The script will then use the string to greet the user by name. name.
002 - Process It’s It’s important to understand that when you run a script, it’s it’s being treated as its own process. You You may have many processes p rocesses running at the same time on your machine, and each has their own unique identification called a Process ID (pid). We’ll look at how we can pass over responsibility to a separate process later on, o n, but for now, here is how we can output the Process ID.
examples/002/process.js console. console .log log( (`This process is pid ${process ${process. .pid pid}` }`) );
Here is the output. This process is pid 5436
How about adding a callback when the current process has ended? So when the script exits, this will fire. process.on process. on( ('exit' 'exit', , (code) => { console. console .log log( (`The process has now finished, exiting with code: ${code ${code}` }`) ); });
Here is the output. The process has now finished, exiting with code: 0
Now let’s let’s looks into standard input (stdin) and standard output (stdout) streams.
stdin & stdout The standard input (stdin) refers to when the user types into the terminal and then submits the data - or chunk to be processed. So the process is reading information from you. The standard output (stdout) is what is returned back to you, the user. To output something using standard output, we use is a simpler version of remember to add console.log
\n
. When using
, we have to
process.stdout.write
at the end of the string to signify a line break, as unlike
, it does not do that for us.
Let’s Let’s make use of some console.log
console.log
. This method
process.stdout.write
process
. We’ll begin with
object methods. We’ll also compare it to using
process.stdout
.
process. process .stdout stdout. .write write( ('Hello, I am writing to standard output\n') output\n'); process. process .stdout stdout. .write write( (`Current working directory: ${process ${process. .cwd cwd() ()}\n` }\n`) ); console. console .log log( (`This script has been running for ${process ${process. .uptime uptime() ()} } seconds`) seconds`);
Here is the output. $ node examples/002/process.js Hello, I am writing to standard output Current working directory: /Users/shaun/Workspace/nobot-examples /Users/shaun/Workspace/nobot-examples This script has been running for 0.064 seconds
Alright, it’s it’s time to use standard input. We’ll We’ll start by asking the user - using the standard output - to type something. We We will then apply UTF-8 so it applies the correct character encoding. To read input from the user, we start by listening to the
stdin
readable
event
listener. listener. This event fires when the user presses presse s enter. When the event fires, in our event listener, we can use
process.stdin.read
to read the chunk of input that the user
has typed. Finally, Finally, we check if the input chunk is not null, and in that case we output it to the user and exit the process. process. process .stdout stdout. .write write( ('Type something then hit enter: \n') \n'); process. process .stdin stdin. .setEncoding setEncoding( ('utf8' 'utf8') ); process.stdin process. stdin. .on on( ('readable' 'readable', , () => { const chunk = process process. .stdin stdin. .read read() (); ; if (chunk !== null) { process. process .stdout stdout. .write write( (`You wrote: ${chunk ${chunk}` }`) ); process. process .exit exit( (0); } });
Here is the output. $ node examples/002/process.js Type something then hit enter: Hello! You wrote: Hello! The process has now finished, exiting with code: 0
Summary In this section we’ve learned how to use standard input and standard output. We were able to do this using the global object
process
, which provides information
about and control over the current process, such as reading standard input, writing output, and retrieving metrics like uptime and the Process ID.
003 – Query String I’m going to assume that your company makes use of a project management tool such as Jira or Target Target Process. Our application will want to interact with the APIs of these tools when it wants to inspect a ticket, or add a comment based on certain business rules. Web based APIs (such as Jira’s Jira’s JQL search feature) may require passing GET parameters. These parameters are appended to the endpoint we are targeting, in what is called the Query String. They are formatted as follows:
etc.
?param1=value1¶m2=value2
We could write a function that would take an object with the key-value pairs for the Query String and build it ourselves, but lucky for us, Node’s standard library has a module called
querystring
, which does just that. All we need to do is pass an
object of key-value pairs (representing the Query String parameters) to the querystring.stringify
string.
method, and it will return our arguments arguments formatted as a query
API is an abbreviation for Application Programming Interface. Company (A) may want to share their product items with another company (B). Company (A) decide to use an API to which Company (B) can make a request to retrieve the product details. This API would allow Company (B) access only if they are authorised. It is down to the API implementation to set these rules. One of the ways to access ticket information in Jira is to use Basic Authentication, which combines your username and password and then encodes it with base64. There Ther e are alternatives, but that’s outside the scope of this book.
examples/003/build-querystring.js const querystring = require require( ('querystring' 'querystring') ); // https://jira.my https://jira.my-company.com/re -company.com/rest/api/latest/s st/api/latest/search?jql="assi earch?jql="assignee=shaun.ston gnee=shaun.stone&startAt=2&max e&startAt=2&maxResults=2" Results=2"
const apiHost = 'https://jira.m 'https://jira.my-company.com/r y-company.com/rest/api/latest/ est/api/latest/search?jql=' search?jql='; const jqlParams = {
assignee: 'shaun.stone' assignee: 'shaun.stone', , startAt: startAt : 2, maxResults: maxResults : 2 }; const apiUrl = `${ `${apiHost apiHost}"${ }"${querystring querystring. .stringify stringify(jqlParams) (jqlParams)}"` }"`; ;
console. console .log log( (`My JQL api call is: ${apiUrl ${apiUrl}` }`) );
Here is the output. $ node examples/003/ examples/003/build-querystrin build-querystring.js g.js My JQL api call is: https://jira.my-company.com/rest/api/late https://jira.my-company.com/rest/api/latest/search? st/search?
jql="assignee=shaun.stone&start jql="assignee=s haun.stone&startAt=2&maxResults At=2&maxResults=2" =2"
You can alternatively do the reverse and create an object from a query string, as the following example illustrates.
examples/003/parse-querystring.js const querystring = require require( ('querystring' 'querystring') ); const url = 'http://www.op 'http://www.opencanvas.co.uk? encanvas.co.uk?myName=Shaun&my myName=Shaun&myAge=28&comment= Age=28&comment=Yes+I+am+gettin Yes+I+am+getting+old' g+old'; const parsedUrl = querystring querystring. .parse parse( (url url. .substring substring( (url url. .indexOf indexOf( ('?' '?') ) + 1)) )); ;
console.log console. log( (`Hi my name is ${parsedUrl ${parsedUrl. .myName myName}` }`) ); console. console .log log( (`I am ${parsedUrl ${parsedUrl. .myAge myAge}` }`) ); console. console .log log( (`Oh and... ${parsedUrl ${parsedUrl. .comment comment}` }`) );
Here is the output. $ node examples/003/ examples/003/parse-querystrin parse-querystring.js g.js Hi my name is Shaun I am 28 Oh and... Yes I am getting old
Summary Dealing with a variety of APIs will give us a lot of control with automating tasks. When constructing a URL with a query string, this will help us with formatting.
004 – URL If you need to break down a URL, consider using the
url
module.
examples/004/url.js const url = require require( ('url' 'url') ); const args = process process. .argv argv. .slice slice( (2); const [urlEntered] = args args; ; if (urlEntered === undefined) {
console.error console. error( ('Please pass a URL e.g. https://www.google.co.uk/search?q=stra https://www.google.co.uk/search?q=stranger+things' nger+things'); process. process .exit exit( (0); } const {
protocol, protocol, slashes slashes, , host host, , query query, , href } = url url. .parse parse(urlEntered) (urlEntered); ; console.log console. log( (`Using protocol: ${protocol ${protocol}` }`) ); console. console .log log( (`Using slashes: ${slashes ${slashes}` }`) ); console. console .log log( (`Host: ${host ${host}` }`) ); console. console .log log( (`Query: ${query ${query}` }`) ); console. console .log log( (`HREF: ${href ${href}` }`) );
Here is the output. $ node examples/004/ examples/004/url.js url.js "https://www.g "https://www.google.co.uk/sea oogle.co.uk/search?q=stranger+ rch?q=stranger+things" things" Using protocol: https: Using slashes: true Host: www.google.co. www.google.co.uk uk Query: q=stranger+th q=stranger+things ings HREF: https://www.go https://www.google.co.uk/searc ogle.co.uk/search?q=stranger+th h?q=stranger+things ings
At this point, once you have parsed the URL, you could use the
querystring
module explained in the previous chapter to parse the query string down from a string to an object of key value pairs.
Summary When we want to break down URLs and extract specific segments, the module can help us do that with ease.
url
005 - OS Node provides us with an
os
module that allows us to dig into the hardware
specifications of our machine. When building my tool, I defaulted to installing it in the home directory, and the good thing about the
os
module, is that you can
get a reference to the home directory path for Windows, Mac or Linux by doing the following.
examples/005/os.js const os = require require( ('os' 'os') ); const homeDirectory = os os. .homedir homedir() (); ;
console. console .log log( (`Your home directory is: ${homeDirectory ${homeDirectory}` }`) );
Here is the output: Your home directory is: C:\Users\shaun
If you need to identify your OS platform to perform separate tasks for Windows and Mac respectively, you can do so by calling the const osPlatform = os os. .platform platform() (); ;
console. console .log log( (`The OS platform is: ${osPlatform ${osPlatform}` }`) );
Here is the output:
platform
function.
The OS platform is: win32
If you’d like to identify the CPU installed, this could be a useful tool for high performance computing when trying to distribute computation between more than one core. Or… you could just show off your specs.
examples/005/os.js const cpuCores = os os. .cpus cpus() (); ; const coreCount = cpuCores cpuCores. .length length; ; const cpuModel = cpuCores[ cpuCores[0 0]. ].model model; ;
console. console .log log( (`I can see your ${cpuModel ${cpuModel} } has ${coreCount ${coreCount} } cores.`) cores.`);
Here is the output: I can see your Intel(R) Core(TM) i7-6700K CPU @ 4.00GHz has 8 cores
Yes, I did recently get an upgrade (cheeky wink). No, I didn’t include this example so you would know… okay I did, I’m not ashamed. The example output below is from a Mac. $ node examples/005/ examples/005/os.js os.js Your home directory is: /Users/shaun The OS platform is: darwin I can see your Intel(R) Core(TM) i5-7360U CPU @ 2.30GHz has 4 cores.
Nope… not as impressive.
Summary Mac and Windows file systems are different, and this could be problematic when you want to build a cross-platform tool. Fortunately, Fortunately, there is a new ne w package that can handle this more elegantly. elegantly. We We will make use us e of it in future examples.
006 - Open Rumour has it that not everyone uses the same operating system. Different CLIs have different commands. For example, the command to list the content of a directory on Unix-based operating systems is
ls
, whereas on Windows it’s
dir
In this example I will show you how we can support both Windows and Mac when opening a browser with a URL using the
module shown in the
os
previous recipe. This time though we need to spawn a new child process to handle the execution of opening the browser. Creating a new sub process requires enough memory in which both the child process and the current program can be executed.
Child Processes A process is an instance of a computer program running on your operating system. It contains the program code and its activity. activity. A child process is a process created by a parent process. Node allows you to create a child process in four different ways:
spawn
,
fork
,
exec
and
execFile
.
.
The
spawn
function launches a command in in a new process, and we can pass
it command arguments. It’s good to note that
spawn
does not create a shell to
execute the command we pass it, making it more efficient than exec
exec
. Also
buffers the command’s command’s generated output and passes all of the output to
a callback function. Use opposed to
spawn
exec
if you know the data returned back is small as
that streams back data in chunks. Use
execFile
if you want
to execute a file without using a shell (apart from Windows where some files cannot be executed). Lastly
fork
is a variation of spawn, the difference is
that it invokes a specified module with an established communication channel that allows messages to be passed back and forth between parent and child.
examples/006/open.js const { platform } = require require( ('os' 'os') ); const { exec } = require require( ('child_process' 'child_process') );
const WINDOWS_PLATFO WINDOWS_PLATFORM RM = 'win32' 'win32'; ; const osPlatform = platform platform() (); ; const args = process process. .argv argv. .slice slice( (2); const [url] = args args; ;
By identifying the platform, we can alternate which command to execute. The user can pass the URL they want to visit. Once we have done some conditional logic to set the command, the sub process can be executed. If a URL is not passed, the script gets aborted by exiting the process.
Please note: Ensure you have installed Google Chrome for this script to work as expected. if (url === undefined) {
console.error console. error( ('Please enter a URL, e.g. "http://www.op "http://www.opencanvas.co.uk" encanvas.co.uk"' '); process. process .exit exit( (0); } let command command; ; if (osPlatform === WINDOWS_PLATFO WINDOWS_PLATFORM) RM) {
command = `start microsoft-edge microsoft-edge:${ :${url url}` }`; ; } else { command = `open -a "Google Chrome" ${url ${url}` }`; ; } console. console .log log( (`executing command: ${command ${command}` }`) ); exec(command) exec (command); ; // On windows machine, Edge browser opens URL // On mac machines, Chrome browser opens URL
Here is the output. $ node examples/006/ examples/006/open.js open.js "http://www.opencanvas.co.uk" "http://www.opencanvas.co.uk" executing command: open -a "Google Chrome" http://www.opencanvas.co.uk http://www.opencanvas.co.uk
Summary This script could be extended to support other platforms if necessary. It could be useful for opening URLs once you have committed your work and pushed to GitHub for example. The tool could provide you with the pull request link and ask if you want to open your browser to see the file changes before merging your pull request.
007 - File System Using the file system to handle files will be one of the key aspects of our build tool. Luckily this comes out of the box with Node. You’ll You’ll most likely have to read in data from configuration files, write new or update upda te existing files. With the projects I worked on, there was a configuration file in JSON format, and some of the values needed to be replaced. I needed to update the ID, the date range, what service requests to call and so on. Here is a watered-down example of a configuration file for a game campaign.
examples/007/data/example-config.json { "projectId": 234789 "projectId": 234789, , "game": "game" : "january-2019-wheres-wallis", "startDate": "startDate" : "2019-01-01T00:00:00" "2019-01-01T00:00:00", , "endDate": "endDate" : "2019-01-08T00:00:00" }
When you are working on some form of template that you want to be configurable to the scope of your project, using a separate JSON file that contains data separated from the source code can be a good approach to take. We want to obtain these values in a script, and we could do something like the following:
Please note: This is a unrecommended example of reading JSON.
examples/007/bad-example/read-json.js const fs = require require( ('fs' 'fs') ); const readJson = file => new Promise Promise((resolve ((resolve, , reject) => {
fs. fs .readFile readFile(file (file, , { encoding encoding: : 'UTF-8' }, }, (err (err, , data) => { if (err) { reject(err) reject (err); ; } resolve( resolve (JSON JSON. .parse parse(data)) (data)); ; }); }); // Usage
readJson(`${ readJson( __dirname}/../data/example-config.json`) __dirname}/../data/example-config.json` .then then(config (config => console console. .log log( (config config. .projectId projectId)) )); ;
Here is the output. $ node examples/007/ examples/007/bad-example/read bad-example/read-json.js -json.js 234789
But… there is no need. In the same way that we use the
require
function to
import a JavaScript file, it’s possible to require JSON files as well. And as an added bonus, it’s automatically parsed as well. It’s as straightforward as this code:
examples/007/read-json.js const { projectId projectId, , startDate startDate, , endDate } = require require( ('./data/example-config');
console.log console. log( (`The ID of the project is: ${projectId ${projectId}` }`) ); console. console .log log( (`The start date of the project is: ${startDate ${startDate}` }`) ); console. console .log log( (`The end date of the project is: ${endDate ${endDate}` }`) );
Here is the resulting output.
$ node examples/007/read-json.js examples/007/read-json.js The ID of the project is: 234789 The start date of the project is: 2019-01-01T00:00:00 2019-01-01T00:00:00 The end date of the project is: 2019-01-08T00:00:00
Okay, Okay, so here’s the scenario. The values you need to place into the configuration file actually exists in a Jira ticket. So instead of updating the values va lues manually, manually, you could pass a ticket/issue number to the script, and it would go and fetch the values and then use them to overwrite the values in the configuration file.
Please note: In this example we will use mock data from examples/007/data/mock-jira-data.json
for brevity. brevity. In a real world example, example, we would do
something like a HTTP request to an API. This will be done later on in part 2.
examples/007/helpers/get-jira-data.js const jiraData = require require( ('../data/mock-jira-data'); // Imagine this data being retrieved from Jira and transformed
const fetchDataFromJ fetchDataFromJira ira = ticketNumber => jiraData[ticke jiraData[ticketNumber] tNumber]; ;
module. module .exports = fetchDataFromJ fetchDataFromJira ira; ;
When creating an object and writing to a file, we first have to convert it to string form by using
JSON.stringify
.
This helper will deal with writing to configuration files. f iles.
examples/007/helpers/write-json.js const fs = require require( ('fs' 'fs') ); const JSON_WHITESPAC JSON_WHITESPACE E = 4; const writeJson = (file (file, , contents) => new Promise Promise((resolve ((resolve, , reject) => {
fs. fs .writeFile writeFile(file (file, , JSON JSON. .stringify stringify(contents (contents, , null, JSON_WHITESPAC JSON_WHITESPACE) E), , (err) => { if (err) { reject(err) reject (err); ; } resolve( resolve (`${ `${file file} } written`) written`); }); }); module. module .exports = writeJson writeJson; ;
Now using the Jira data, let’s let’s create a configuration conf iguration file.
examples/007/write-config.js const path = require require( ('path' 'path') ); // helpers
const writeJson = require require( ('./helpers/write-json' './helpers/write-json') ); const getJiraData = require require( ('./helpers/get-jira-data'); const args = process process. .argv argv. .slice slice( (2); const [ticket] = args args; ; const CONFIG_FILE = 'config.json' 'config.json'; ; const jiraTicket = ticket || 'GS-1000' 'GS-1000'; ; const jiraData = getJiraData getJiraData(jiraTicket) (jiraTicket); ; if (jiraData === undefined) {
console.log console. log( (`JIRA ticket ${jiraTicket ${jiraTicket} } not found`) found`); process. process .exit exit( (0); }
const newConfigFile = path path. .join join(__dirname (__dirname, , 'data' 'data', , CONFIG_FILE) CONFIG_FILE); ;
writeJson(newConfigFile, writeJson(newConfigFile , jiraData) .then then(msg (msg => console console. .log log(msg)) (msg)) .catch catch((err) ((err) => { throw err err; ; });
You should notice that the Jira ticket number can be passed by the user u ser.. Hopefully this is building up a picture for you. $ node examples/007/write-config.js examples/007/write-config.js "GS-1000" /Users/shaun/Workspace/nobot-ex /Users/shaun/Wo rkspace/nobot-examples/examples amples/examples/007/data/confi /007/data/config.json g.json written
Try it for yourself! The available mock Jira ticket numbers defined in mock-jira-data.json
are: ‘GS-1000’, ‘GS-1005’, ‘GS-1007’ and ‘GS-1020’.
path module allows you to work with file paths in Node. Each operating system uses a different file path p ath separator. If you are on a Windows machine, you’ll find that it uses backslashes, whereas Unix-like Operating systems like Mac use forward slashes. To To avoid paths not resolving, res olving, we can make use of the
path.join
function to join segments of a URL. To use this function, function, we
simply pass to it the segments of the path, and it returns to us the built path appropriate to our operating system.
Making use of
path
is good practice, but what if we want to know the current
directory path dynamically?
__dirname This is a global variable holding the full path to the current directory.. When used in conjunction with directory
path.join
files and/or directories in our current directory.
, it allows us to create new
Summary Writing and updating configuration files should be as automated as possible. If you have the data sitting on a ticket somewhere and it needs to be pulled into your project, why do it manually? It can also minimise the risk of data entry mistakes. You You just need to make sure the person entering the data in the ticket knows what he is doing.
008 - Zip Up until now we have been making use of Node’s standard API library such as the
,
process
path
and
os
modules. If we want to extend our choice to use more
libraries that provide further functionality f unctionality,, we can turn to npm. The npm ecosystem is huge, and if you do a search for a library with certain functionality, functionality, chances are you’ll find it. Here you can see the most used packages https://www.npmjs.com/browse/depended.. Feel free to explore what is available! https://www.npmjs.com/browse/depended The use case here is as follows: a member of your team has requested that you zip up a text file and an image. To accomplish this, we will use an external dependency dependency identified as
archiver
.
Please note: As you might recall from the Technical Overview section, to install this dependency, you’d normally need to run
npm install archiver
in the terminal.
This command will install the dependency into node_modules and add it to the project’s project’s dependencies in its this now, as the
nobot-examples
dependencies listed in its
package.json
file. However, However, you will will not need to do
project that you are using already has these package.json
, and you have run
npm install
before you
started (and if you haven’t, there is no time like the present). As a reminder, npm install
looks at your project’s project’s
package.json
file and installs all the dependencies
listed in it. To use the
archiver
npm module, after it’ it’ss installed, all we need to do is require it,
ust like we’ve been doing with the Node’s standard library modules. The require method will look in the node_modules and find a directory called archiver and use the source code found inside this directory.
Streams are a pattern that makes a conversion of huge operations and breaks it down into manageable chunks. If you were eating a big birthday cake, you wouldn’t try to scoff the entire thing into your mouth, you would instead cut small slices. Or pick at it with your fingers without anyone noticing you’ve had so much already. Yes, Vijay. I did see you do it… many times.
examples/008/zip.js const archiver = require require( ('archiver' 'archiver') );
So just to recap, if we didn’t have this package installed, Node would throw an error saying that this module does not exist; but we have, so it won’t give us an earful. For this example we’ll also need to use the
fs
and
path
modules from Node’s Node’s
standard library, so let’s let’s require these as well. const fs = require require( ('fs' 'fs') ); const path = require require( ('path' 'path') );
First off, there has to be a write stream to a filename of our choice. Then we set up our archiver, declaring it will be a zip file, and the compression method will be
zlib
- a library used for data compression.
Please note: The idea of streams will be covered in a later chapter. const ZLIB_BEST_COMP ZLIB_BEST_COMPRESSION RESSION = 9; // create a file to stream archive data to.
const zipPath = path path. .join join(__dirname (__dirname, , 'files.zip' 'files.zip') ); const output = fs fs. .createWriteStream createWriteStream(zipPath) (zipPath); ; const archive = archiver archiver( ('zip' 'zip', , {
zlib: zlib : { level level: : Z_BEST_COMPRES Z_BEST_COMPRESSION SION } });
When the file stream has closed, let’s run a callback that logs out the total bytes and informs the user that the archiving has completed. // listen for all archive data to be written
output.on output. on( ('close' 'close', , () => { console. console .log log( (`Total bytes: ${archive ${archive. .pointer pointer() ()}` }`) ); console. console .log log( ('archiving has now finished.') finished.'); });
If an error is to occur, we’ll need to catch it in another callback and throw the error so we can identify what went wrong. // good practice to catch this error explicitly
archive.on archive. on( ('error' 'error', , (err) => { throw err err; ; });
Finally, Finally, we pipe the file stream to the archiver, and append the files we would like to add, then finalise the operation. The
append
function is taking in a read
stream from both the text file and our image, so that when we open the zip, we should see the files inside. archive. archive .pipe pipe(output) (output); ; // add files (read the copy.txt and logo.jpg and output with different names)
const textPath = path path. .join join(__dirname (__dirname, , 'copy.txt' 'copy.txt') ); const logoPath = path path. .join join(__dirname (__dirname, , 'logo.jpg' 'logo.jpg') );
archive.append archive. append( (fs fs. .createReadStream createReadStream(textPath) (textPath), , { name name: : 'content.txt' }); archive. archive .append append( (fs fs. .createReadStream createReadStream(logoPath) (logoPath), , { name: name: 'nobot.jpg' }); // finalize the archive (ie we are done appending files but streams have to finish yet)
archive. archive .finalize finalize() (); ;
Here is the output. $ node examples/008/ examples/008/zip.js zip.js Total bytes: 105156 archiving has now finished.
Summary I managed to reduce the file size by about a half. How awesome is that? If you need to send large files over email or need to transfer to external media, then it would be ideal to have a script to zip it up for you beforehand.
009 - Colours Yes, I am from the UK, and that is why I have to be so blooming awkward and spell colours like I do. When we log our output, it can be useful to differentiate what type of message is being displayed to the user u ser.. We We can do that using the npm
colors
package, by giving giving our output a different colour based on the type of
message we are displaying. If something went wrong; use red, if something went right; use green. If the user needs to be warned about something; s omething; use yellow. yellow. Let’s give it a try.
examples/009/log-color.js examples/009/log-color .js require( require ('colors' 'colors') );
You will see when I require r equire
colors
at the top, there is is no need to assign it it to a
variable or constant, because once it’s loaded in, the library takes effect - it extends
String.prototype
.
console.log console. log( ('This is a success message'. message'.green green) ); console. console .log log( ('This is a warning message'. message'.yellow yellow) ); console. console .log log( ('This is a error message'. message'.red red) );
The
colors
package gives us a handful of colours that can be applied to any
string. This allows us to give semantic feedback to the user us er..
To take this even further, it would be good to have a helper function which outputs messages in a specific format and colour based on the type of message. To do that, let’s start by creating a module with constants cons tants for the message types.
examples/009/constants/message-types.js const MESSAGE_TYPES = {
SUCCESS: 'success' SUCCESS: 'success', , WARNING: WARNING : 'warning' 'warning', , ERROR: ERROR : 'error' }; module. module .exports = MESSAGE_TYPES MESSAGE_TYPES; ;
Now let’s let’s create the module with the logging helper function. Our helper function will accept two arguments: the message and the message type (optional). It will construct the formatted and coloured message and invoke console.log
to output the message message to the screen.
We need to include the for two script files.
MESSAGE_TYPES
constants in the helper as they will be needed
examples/009/helpers/log.js require('colors' require( 'colors') ); const { ERROR ERROR, , WARNING WARNING, , SUCCESS } = require require( ('../constants/message-types'); module.exports = function log module. log(message (message, , type) { let colorMessage colorMessage; ; switch (type) { case ERROR ERROR: : colorMessage = `[ERROR] ${message ${message. .red red}` }`; ; break; case WARNING WARNING: : colorMessage = `[WARNING] ${message ${message. .yellow yellow}` }`; ; break; case SUCCESS SUCCESS: : colorMessage = `[SUCCESS] ${message ${message. .green green}` }`; ; break; default: default : colorMessage = `[INFO] ${message ${message}` }`; ; } console. console .log log(colorMessage) (colorMessage); ; };
Now we can use the log function by requiring it at the top and calling it with the message and its type.
examples/009/colors.js const log = require require( ('./helpers/log' './helpers/log') ); const { ERROR ERROR, , WARNING WARNING, , SUCCESS } = require require( ('./constants/message-types');
log('This log( log( log ('This log( log ('This log( log ('This
is is is is
a success message', message', SUCCESS) SUCCESS); ; a warning message', message', WARNING) WARNING); ; a error message', message', ERROR) ERROR); ; an info message') message');
The output demonstrated below will not show the colours used, but when you run it yourself, you will see it. $ node examples/009/colors.js [SUCCESS] This is a success message [WARNING] This is a warning message [ERROR] This is a error message [INFO] This is an info message
Summary It goes without saying that using this in your application gives clearer feedback, thus makes it more user-friendly.
010 - Question Node has a native module known as
readline
, which builds upon the stdin and
stdout, and provides an interface – or wrapper - between them. It allows us to read one line at a time, which means we can easily prompt and save answers provided by the user. Let’s Let’s see if we can make use of this module to create a project directory based on the user’s input. For this project we’ll need the file system module, the
readline
module, the
standard input & output from the process object, and the path module. All native modules to Node.
examples/010/question.js const fs = require require( ('fs' 'fs') ); const readline = require require( ('readline' 'readline') ); const { stdin stdin, , stdout } = require require( ('process' 'process') ); const path = require require( ('path' 'path') );
First, let’s let’s set up the interface that links standard input and output. const interfaceInsta interfaceInstance nce = readline readline. .createInterface createInterface(stdin (stdin, , stdout) stdout); ;
Next, we’ll use the interface’s question method to ask the user a question (output) and link the callback function which will deal with the user ’s answer
(input). interfaceInstance. interfaceInstance .question question( ('What is the name of your project? ', ', onProjectInpu onProjectInput) t); ;
Now we need to define our callback function. Let’s Let’s add it at the top, right after the require statements. We’ll We’ll make it a concise function. As it’s always good practice to clean up after af ter yourself, the function will close the interface and destroy the standard input so nothing else can be inputted. Finally we call another function with the user input (the project name), which will deal with creating the project directory and handling errors. const onProjectInput = (name) => {
interfaceInstance.close interfaceInstance. close() (); ; stdin. stdin .destroy destroy() (); ; createProjectDirectory(name) createProjectDirectory (name); ; };
Right after this function, let’s define the
createProjectDirectory
function which we are
calling. This function receives the user ’s input; the desired project name. It starts by trimming the input to get rid of leading and trailing spaces. It then does some error handling, to prevent attempting to create a directory without a name or for a directory that already exists. Of course in a real-world situation, this would need to be stricter, which a regular expression could help with. For our toy example, we’ll stick with our naive error handling. Once we’re happy with the input, we go ahead and create the directory using fs.mkdirSync
. This function will create a directory synchronously. synchronously.
const createProjectD createProjectDirectory irectory = (enteredName) => { const name = enteredName enteredName. .trim trim() (); ; if (name === '' '') ) {
console.log console. log( ('Cannot create a project without a name') name'); process. process .exit exit( (0); } const projectPath = path path. .join join(__dirname (__dirname, , name) name); ; if ( (fs fs. .existsSync existsSync(projectPath)) (projectPath)) {
console.log console. log( (`${ `${name name} } already exists`) exists`); process. process .exit exit( (0);
} console. console .log log( (`creating a new project called ${name ${name}` }`) ); fs. fs .mkdirSync mkdirSync(projectPath) (projectPath); ; };
What does this look like in action? $ node examples/010/question.js What is the name of your project? yogi-bear creating a new project called yogi-bear
Summary The
readline
module is fantastic for creating creating step-by-step feedback for the user. user.
When creating our CLI application, we want to make it interactive, like a questionnaire in case the user us er didn’t provide the initial arguments. It makes it more user-friendly. user-friendly. Because it’s good to be friendly!
011 – Cloning In this chapter we’re going to learn how to programmatically clone repositories. Here it comes, a new dependency. dependency. This one is called
shelljs
, and allows us to
execute commands on the CLI through our scripts. We require it at the top of our script, as a s well as the path
module and a pre-made
config.json
colors
package, the native
file containing containing URLs URLs of the repositories
we would like to clone.
Please note: As mentioned earlier in the book, you will find a This will need to be copied and created as
config.json
config.example.json
.
. The reasoning behind this
is that some examples will have sensitive data that you don’t want to commit and push to a public repository r epository.. It is especially important when we deal with the email and SMS examples later in this book. In your own
config.json
file, you can add as many repositories as you’d like to the
array, array, and then run the script to see them be cloned.
examples/011/clone-repositories.js examples/011/clone-repositories.js require('colors' require( 'colors') ); const path = require require( ('path' 'path') ); const shell = require require( ('shelljs' 'shelljs') ); const { repositories } = require require( ('./config' './config') );
const repositoriesDi repositoriesDirectory rectory = path path. .join join(__dirname (__dirname, , 'my-repositories' 'my-repositories') );
Here is the output of the repositories directory path. /Users/shaun/Workspace/nobot-ex /Users/shaun/Wo rkspace/nobot-examples/examples amples/examples/011/my-reposit /011/my-repositories ories
Our
config.json
file looks like this:
examples/011/config.json examples/011/config.json { "repositories": [ "repositories": "https://github.com/smks/nobot-repo-1", "https://github.com/smks/nobot-repo-2" ] }
A function can now be constructed to take a destination path and an array arr ay of repositories, loop through each of the repositories, and use the execute
git clone
shelljs
module to
on each of them. The repositories will be cloned into into the ‘my-
repositories’ directory. We change into this directory initially.
examples/011/clone-repositories.js examples/011/clone-repositories.js function cloneRepositories cloneRepositories(repositoryPath (repositoryPath, , repositoryList = []) { const repositoryCount = repositoryList repositoryList. .length length; ; if ( (! !repositoryPath || repositoryCou repositoryCount nt === 0) {
console.log console. log( ('Invalid path or repository list') list'); return; } console. console .log log( (`Cloning repositories to: ${repositoriesDirectory ${repositoriesDirectory}` }`. .blue blue) ); shell. shell .cd cd(repositoryPath) (repositoryPath); ; repositoryList.forEach repositoryList. forEach((repositoryUrl ((repositoryUrl, , index) => { console. console .log log( (`Cloning ${index ${index + 1} of ${repositoryCount ${repositoryCount}` }`. .cyan cyan) ); shell. shell .exec exec( (`git clone ${repositoryUrl ${repositoryUrl} } --progress -b master`) master`); });
console. console .log log( ('Completed cloning of repositories' repositories'. .green green) ); }
We can then call this function with the
repositoriesDirectory
the repositories array which we extracted from the
we defined above and
config.json
file.
cloneRepositories(repositoriesDirectory cloneRepositories (repositoriesDirectory, , repositories) repositories); ;
This script can be used to prepare your build tool for use and would run as a post install script. So before you start releasing games, you want to ensure that you have the templates of the games, and the website you want to deploy to readily available. $ node examples/011/ examples/011/clone-repositori clone-repositories.js es.js Cloning repositories to: /Users/shaun/ /Users/shaun/Workspace/nobot Workspace/nobot-examples/examp -examples/examples/011/my-repo les/011/my-repositories sitories Cloning 1 of 2 Cloning into 'nobot-repo-1'. 'nobot-repo-1'... .. remote: Counting objects: 3, done. remote: Compressing objects: 100% (2/2), done. remote: Total 3 (delta 0), reused 3 (delta 0), pack-reused 0 Cloning 2 of 2 Cloning into 'nobot-repo-2'. 'nobot-repo-2'... .. remote: Counting objects: 3, done. remote: Compressing objects: 100% (2/2), done. remote: Total 3 (delta 0), reused 3 (delta 0), pack-reused 0 Completed cloning of repositories
Summary When we move on to using the build tool, we will make use of external repositories. One of our repositories will be a website, and another will be a library of game templates we can build from. This process will help us clone repositories that we depend on to deploy games. Once we’ve cloned them, we can then look into doing a use of it. By doing a stable versions.
git pull
git pull
on a repository whenever we want to make
on the templates, templates, we know we’ve got the latest
012 – Branching If you are working on a repository for a project, and you need to implement a new feature, the first thing you usually do is grab the latest changes from the master
or
develop
branch. Once all all these changes have been pulled in, you then then
create a feature branch. Likely your branch will have the same identifier as the ticket you are working on. So I may have a ticket identified as
MARKETING-248
and it
makes sense to have a branch to match. Project management tools – if setup correctly – can integrate with git branches for better visibility such as commits and merges. We are going go ing to clone the
nobot-repo-1
repository from the the previous chapter into into
the ‘012’ directory, directory, and create a feature branch from the base branch e.g.
master
.
The only question we will ask the user is, ‘What is the ticket ID?’.
Please note: In the
config.json
file you should provide your own forked version of
the repository so you have no problems with permissions. Our
config.json
file looks like this:
examples/012/config.json { "repository": { "repository": "delivery": "delivery" : "https://github.com/smks/nobot-repo-1", "baseBranch": "baseBranch" : "master" } }
Start by running the setup script.
examples/012/setup.js require('colors' require( 'colors') ); const shell = require require( ('shelljs' 'shelljs') ); const { repository } = require require( ('./config' './config') ); const { delivery } = repository repository; ;
console. console .log log( (`Cloning ${delivery ${delivery}` }`. .cyan cyan) ); shell. shell .cd cd(__dirname) (__dirname); ; shell. shell .exec exec( (`git clone ${delivery ${delivery} } --progress`) --progress`);
This will clone the repository as seen in the source code above.
Please note: This needs to be run as a prerequisite before we can create a new branch. $ node examples/012/ examples/012/setup.js setup.js Cloning https://github. https://github.com/smks/nobotcom/smks/nobot-repo-1 repo-1 Cloning into 'nobot-repo-1'. 'nobot-repo-1'... .. remote: Counting objects: 3, done. remote: Compressing objects: 100% (2/2), done. remote: Total 3 (delta 0), reused 3 (delta 0), pack-reused 0
In this example we will be making use of a synchronous version of the native readline
module provided by Node. This module is installed via npm and is
identified as
readline-sync
. We We want to use this module to stop execution until the
user has inputted the data we are expecting.
Synchronous and Asynchronous If something is synchronous, then the code execution will block (or wait) for the execution to return before continuing. In our case, when creating a directory, we will wait for the directory to be created before moving onto the next operation. Asynchronous in the context of Node will execute the code, but will not block or wait for a result back. In the majority of cases you will get back the result of what you have done as an argument to a callback. Now the repository exists, we are going to create a script that will change into the directory, run
git pull
to make sure it’s it’s up to date, and then create a new
branch using the git executable. First we require the npm packages needed to accomplish this functionality. functionality.
examples/012/new-branch.js const shell = require require( ('shelljs' 'shelljs') ); const readLineSync = require require( ('readline-sync' 'readline-sync') ); const path = require require( ('path' 'path') ); const { repository } = require require( ('./config' './config') );
We’re going to need the repo name and the base branch. We can get these from our
config.json
file.
const { delivery delivery, , baseBranch } = repository repository; ; const repoName = delivery delivery. .substring substring( (delivery delivery. .lastIndexOf lastIndexOf( ('/' '/')) )); ;
Using one of the
shelljs
methods
cd
we can change our current working
directory. directory. Our new directory
nobot-repo-1
let’s let’s get the path to
and then
nobot-repo-1
now exists in our current directory, directory, so cd
into it.
// Changing into the repo's directory
const repoPath = path path. .join join(__dirname (__dirname, , repoName) repoName); ;
shell. shell .cd cd(repoPath) (repoPath); ;
Now we are sitting in the directory of
nobot-repo-1
. When we run
git
, it’s in the
scope of this repository. Before we create a branch, we want to make sure we are on the right base branch, and that it’s the correct one to branch from. In this case it’s
master
config.json
, but could be different for you. That’s why we included it in the
file.
// Checkout to base branch
shell. shell .exec exec( (`git checkout ${baseBranch ${baseBranch}` }`) );
Let’s Let’s pull in all the changes from this base branch, just in case someone else has pushed changes to the remote branch. This is good practice when creating a new branch. // Making sure we have the latest changes from the remote origin
shell. shell .exec exec( (`git pull origin ${baseBranch ${baseBranch}` }`) );
Perfect, now we want to create a branch based on the ticket we’re working on. But first let’s prompt the user for it. // prompt for the ticket ID
const ticketId = readLineSync readLineSync. .question question( ('What is the ticket ID? ', ', {
limit: input => input limit: input. .trim trim(). ().length length > 0, limitMessage: limitMessage : 'Please enter a ticket ID (e.g. GOT-123)' });
The
limit
property is a check to to see if the user input meets the the validation
requirements. This could be a regular expression or a function. In this case I used a function which ensures that the user enters more than an empty string. The limitMessage
value will be displayed if the user failed the
We have what we need. Using the a new branch and check it out. // Create a new branch
shell. shell .exec exec( (`git checkout -b ${ticketId ${ticketId}` }`) );
Now let’s run the script. $ node examples/012/new-branch.js examples/012/new-branch.js Already on 'master' Your branch is up-to-date with 'origin/master'. From https://github. https://github.com/smks/nobot-r com/smks/nobot-repo-1 epo-1 * branch master -> FETCH_HEAD Already up-to-date. What is the ticket ID? GS-101 Switched to a new branch 'GS-101'
-b
limit
check.
option we are telling Git we want to create
Summary When working with source control we often need to create feature branches so our work can be done in isolation. We’ve We’ve learned that this too can be automated using Node.
013 – Retain State When we are dealing with the process of a script, and it exits, then all data or state is forgotten. So what should we do if we need to retain state? When you are in a browser, you can make use of something like local storage or cookies, but with a CLI application, we don’t have this. Instead I nstead what we can do, is create hidden files (aka dot-files) to retain - or persist - our data. In Unix-like systems, files prefixed with a dot are hidden by default, which is why they are called dotfiles. I know… what a revelation. r evelation. A great example to demonstrate this would be to store some reminders. We We have one script that adds a reminder, and one that lists the reminders. Ensure you have .reminders.json
present, if not, then run
node setup.js
in the root of the
nobot-examples
repository. repository. For convenience, I created a short list by default.
examples/013/.reminders.json { "reminders": [ "reminders": "get banana from kitchen", kitchen", "put food in fridge", fridge", "go to standup", standup", "commit work" ] }
Yes, I do forget some of these things when I arrive to the office. Let’s Let’s begin with the script that deals with outputting the stored list. It’s It’s simply reading the contents of the dot-file which stores our reminders. The two scripts we will be writing soon will share the same constants, so we create a new file called
constants.js
, which our scripts can import. This This is very
good practice for many reasons: it avoids typos, it makes changing the values much easier and less error-prone (as the change only needs to be done in one place), and it makes our code self-documenting whenever we reference these constants. When writing to the JSON file, I am specifying the whitespace to be used using a constant
JSON_WHITESPACE
. This will make the JSON file more human readable
when written back to the file f ile system. With the constant
NO_CHOICE_MADE
choosing what reminder to mark as done, we will get an index of
, when -1
if the user
decided to cancel. These constants will be used soon in our scripts.
examples/013/constants.js module.exports = { module. JSON_WHITESPACE: JSON_WHITESPACE : 4, NO_CHOICE_MADE: NO_CHOICE_MADE : -1 };
Now let’s let’s move on to our first script which lists the reminders we currently have stored.
examples/013/list-reminders.js require('colors' require( 'colors') ); const fs = require require( ('fs' 'fs') ); const readLineSync = require require( ('readline-sync' 'readline-sync') ); const { JSON_WHITESPACE JSON_WHITESPACE, , NO_CHOICE_MAD NO_CHOICE_MADE E } = require require( ('./constants' './constants') );
We want to read in the reminders stored in the JSON file. If none exist, there is no need to show a list of reminders. const { reminders } = require require( ('./.reminders' './.reminders') ); if ( (reminders reminders. .length === 0) {
console.log console. log( ('No reminders!'. reminders!'.green green) ); process. process .exit exit( (0); }
Next thing to do is output the list of reminders which do exist and ask the user to choose one to remove, i.e. mark as done. const index = readLineSync readLineSync. .keyInSelect keyInSelect(reminders (reminders, , 'What reminder have you dealt with? ') '); if (index === NO_CHOICE_MADE NO_CHOICE_MADE) ) {
process. process .exit exit( (0); }
We use the
readline-sync
module here to to let the user choose the reminder to
remove. We We pass the reminders array to
, which in turn
readLineSync.keyInSelect
presents the reminders to the user as numbered options to choose from. The user types in the desired option’s option’s number - meaning they have dealt with that
reminder, or
0
to cancel the operation. When the user has chosen a reminder, reminder, it
gets removed from the array using
splice
, and the reminders file gets overridden
with the updated array. array. console. console .log log( (`you removed '${reminders[index] '${reminders[index]}'` }'`. .red red) ); reminders. reminders .splice splice(index (index, , 1); fs. fs .writeFileSync writeFileSync( (`${ __dirname}/.reminders.json`, __dirname}/.reminders.json` , JSON JSON. .stringify stringify( ({ reminders }, null, JSON_WHITESPA JSON_WHITESPACE)) CE)); ;
But what if we want to add new reminders? That’s what this next script is for fo r. We will show the output at the end of this chapter.
examples/013/add-reminder.js examples/013/add-reminder .js const fs = require require( ('fs' 'fs') ); const { JSON_WHITESPACE } = require require( ('./constants' './constants') ); const { reminders } = require require( ('./.reminders' './.reminders') ); const args = process process. .argv argv. .slice slice( (2); let reminder = args[ args[0 0]; if (reminder === undefined) {
console.log console. log( ("Pass a reminder, e.g. 'pick up rabbit'") rabbit'"); process. process .exit exit( (0); } reminder = reminder reminder. .trim trim() (); ; const hasReminderAlr hasReminderAlready eady = reminders reminders. .indexOf indexOf(reminder) (reminder) > -1 -1; ; if (hasReminderAlr (hasReminderAlready) eady) {
console.log console. log( (`Doh! Already have the reminder '${reminder '${reminder}' }' set`) set`); process. process .exit exit( (0); } reminders. reminders .push push(reminder) (reminder); ;
fs. fs .writeFileSync writeFileSync( (`${ __dirname}/.reminders.json`, __dirname}/.reminders.json` , JSON JSON. .stringify stringify( ({ reminders }, null, JSON_WHITESPA JSON_WHITESPACE)) CE)); ; console. console .log log( ('Yes! Added reminder') reminder');
Now on to the demo. First we will add a reminder. $ node examples/013/ examples/013/add-reminder.js add-reminder.js 'jump around' Yes! Added reminder
Okay I’ve just jumped around. ar ound. Now I need to tick off that reminder. $ node examples/013/list-reminders.js examples/013/list-reminders.js [1] [2] [3] [4] [5] [0]
get banana from kitchen put food in fridge go to standup commit work jump around CANCEL
What reminder have you dealt with? [1...5 / 0]: 5 you removed 'jump around'
Summary If you have a project you want to deploy, deploy, and do not want to re-enter data associated with the deployment, you can retain the state in the root of a specific project.
014 – Choose Template Imagine this scenario. You work for a gaming ga ming company, company, let’s call it Games4U. This company has a few game templates for simple games such as: backgammon, chess, draughts and poker. Each game template acts as the skeleton for the game, and can be used to create a custom game of that type for a client. Your manager has asked you to create a new poker game. You need to make a copy of the poker template, and rename it to the project pro ject name. We We will call it
poker-ten-stars. Usually, Usually, you would navigate to the poker template’s directory, directory, and copy it to another location under a different name. But you find this tedious to do every time, so you decide to write a script to automate it for you. So first thing we do is create a new file called input from the user we include the native
path
readline-sync
. To read
create-game-from-template.js
module. We also include the the
module to resolve paths on the file system, and the the
colors
change the colour of our text. Finally, Finally, we include a new library called which extends the native
fs
capability which the original
module to fs-extra
module and allows us to copy directories - a fs
module doesn’t have.
,
examples/014/create-game-from-template.js examples/014/create-game-fro m-template.js require('colors' require( 'colors') ); const readLineSync = require require( ('readline-sync' 'readline-sync') ); const path = require require( ('path' 'path') ); const fse = require require( ('fs-extra' 'fs-extra') );
If you look in the
014
example, you’ll find a a
game-templates
directory that
contains each of the templates available. They are simplified for demonstration. └─game-templates ├─backgammon ├─chess ├─draughts └─poker
We want to read this directory and return an array of the templates listed, because one of these needs to be copied. First, we want to construct the path to the game templates directory. This would be a string like so: /Users/shaun/Workspace/nobot-ex /Users/shaun/Wo rkspace/nobot-examples/examples amples/examples/014/game-templ /014/game-templates ates
With the path to this directory, directory, we can do a synchronous read on it to return the subdirectories as an array. const GAME_TEMPLATES = 'game-templates' 'game-templates'; ; const NO_CHOICE_MADE = -1 -1; ; // 1. Use a game template already built
const templatesDir = path path. .join join(__dirname (__dirname, , GAME_TEMPLATES GAME_TEMPLATES) ); const templates = fse fse. .readdirSync readdirSync(templatesDir) (templatesDir); ;
Great, we have an array ar ray of strings with the name of each template directory. The good thing about reading the directory is that whatever is added here will reflect as a choice in the array. Using the
keyInSelect
method, we can spit this out to the
user so they can choose one. Here is an example. $ node examples/014/ examples/014/create-game-from create-game-from-template.js -template.js
[1] [2] [3] [4] [0]
backgammon chess draughts poker CANCEL
Choose one from list [1...4 / 0]:
The index of the item chosen is returned as a constant. We We exit if they didn’t choose a template. Choosing 0 returns -1. const index = readLineSync readLineSync. .keyInSelect keyInSelect(templates) (templates); ; if (index === NO_CHOICE_MADE NO_CHOICE_MADE) ) {
process. process .exit exit( (0); }
By this point the user has chosen the template they want to copy from, but we don’t know the name of the new project directory. Based on our requirements, it needs to be called
poker-ten-stars
. Let’s Let’s prompt the user with a question, asking
them what they would like to call the project. // 2. Create a new project reskin based on our template
const projectName = readLineSync readLineSync. .question question( ('What is the name of your game? ', ', {
limit: input => input limit: input. .trim trim(). ().length length > 0, limitMessage: limitMessage : 'The project has to have a name, try again' });
As a sanity check, we show the user the project name they have chosen, and ask if they are ready to proceed with the copy. const confirmCreateD confirmCreateDirectory irectory = readLineSync readLineSync. .keyInYN keyInYN( (`You entered '${projectName '${projectName}', }', create directory with this name?
`);
If the user typed
'y'
, it means we can proceed. First, we grab the chosen
template from the array of templates. We We then construct the path to the template by concatenating the path to the templates directory with the template name. We do something similar with the destinations path by concatenating the path to the current working directory with the chosen project name we inputted. // 3. If happy to create, copy the template to the new location
if (confirmCreateD (confirmCreateDirectory) irectory) { const template = templates[inde templates[index] x]; ; const src = path path. .join join(templatesDir (templatesDir, , template) template); ; const destination = path path. .join join(__dirname (__dirname, , projectName) projectName); ;
fse.copy fse. copy(src (src, , destination) .then then(() (() => console console. .log log( (`Successfully created ${ ${destination destination}` }`. .green green)) )) .catch catch(err (err => console console. .error error(err)) (err)); ; } else { console. console .log log( ('Aborted creating a new game') game'); }
As you can see, we use the arguments. The
copy
copy
method and pass the source and destination destination as
method returns a promise, so
then
gets called when the
copy was successful. If any problems occurred (such as permission errors), they are printed to the console. Here is the entire output: $ node examples/014/ examples/014/create-game-from create-game-from-template.js -template.js [1] [2] [3] [4] [0]
backgammon chess draughts poker CANCEL
Choose one from list [1...4 / 0]: 4 What is the name of your game? poker-ten-stars You entered 'poker-ten-stars', create directory with this name? [y/n]: y Successfully created /Users/shaun/Wo /Users/shaun/Workspace/nobot-e rkspace/nobot-examples/example xamples/examples/014/poker-ten s/014/poker-ten-stars -stars
Summary Here we are providing useful feedback to the user by showing them a list of templates that are available. The user can make a more informed choice of what template they would like to create.
015 - Email Email is a great way to notify colleagues when something has been done. You You may have completed a task and you want to let your team know it’s been processed successfully. Alternatively if something goes wrong, you can trigger an email to provide visibility to the team. If we start thinking about how this build tool will run, we can imagine a server running locally or in the cloud where we can set up a Cron job that will call the build tool at intervals - hourly, daily or weekly - so it can work without us manually initiating it. When this process is running on its own though, we have no idea what’s happening unless we have some sort of feedback. Email is a good way to deal with this. At work, your company most likely has some email system configured using a protocol such as SMTP (Simple Mail Transfer Protocol). Your Your team can also be emailed at once with a group email such as ‘
[email protected]’. We can use this email group as the default defa ult email address for all error messages encountered by the Cron job during the build. Here is an example of a configuration for the SMTP Protocol.
examples/015/config.json
{ "FROM_EMAIL": "Nobot Test
" .com>", , "TO_EMAIL": "TO_EMAIL" : "Game Studios Team " ", "HOST": "HOST" : "mysmtp.domain.io" "mysmtp.domain.io", , "PORT": "PORT" : "2525" "2525", , "AUTH": "AUTH" : { "USERNAME": "USERNAME" : "ENTER_USERNAME" "ENTER_USERNAME", , "PASSWORD": "PASSWORD" : "ENTER_PASSWORD" } }
Now we move on to the script that will send out emails. At the top we load in the config.json
file that holds the SMTP configuration. We We are making use of an
external dependency called
nodemailer
.
examples/015/send-email.js require('colors' require( 'colors') ); const config = require require( ('./config' './config') ); const nodemailer = require require( ('nodemailer' 'nodemailer') );
Our script requires two arguments: the subject and body of the email. If these are not passed, the script will terminate. const args = process process. .argv argv. .slice slice( (2); const REQUIRED_FIELD REQUIRED_FIELDS_COUNT S_COUNT = 2; if ( (args args. .length !== REQUIRED_FIELDS_ REQUIRED_FIELDS_COUNT) COUNT) {
console.log console. log( ( 'Two arguments required: subject and body.'. body.'.red red, , 'E.g. node send-email.js "Where\'s my tea?" "So yeah... where is it?"'. it?"'.cyan ); process. process .exit exit( (0); }
If it passes this conditional check, it means the arguments count is correct. Let’s grab the two arguments and assign ass ign them. While we are at it, let’s let’s extract the configuration values to set up the SMTP transporter. transporter. const [subject [subject, , body] = args args; ; const {
HOST HOST, , PORT PORT, , FROM_EMAIL FROM_EMAIL, , TO_EMAIL } = config config; ; const { USERNAME USERNAME, , PASSWORD } = config config. .AUTH AUTH; ;
Now we pass in the configuration. For the demo we are setting secure to false, but you would of course set it to true. const transporter = nodemailer nodemailer. .createTransport createTransport( ({
host: HOST host: HOST, , port: port : PORT PORT, , secure: secure : false, auth: auth : { user: user : USERNAME USERNAME, , pass: pass : PASSWORD } });
Our transporter will have to send a message object. Here it is. const message = {
from: FROM_EMAIL from: FROM_EMAIL, , to: to : TO_EMAIL TO_EMAIL, , subject, subject, body, body, html: html : `${ `
${body body}
` }` };
Lastly, we use the
sendMail
method on the transporter, transporter, which takes a callback
function to call with error/info at the end of the operation. transporter.sendMail transporter. sendMail(message (message, , (err (err, , info) => { if (err) { console. console .error error( (`Error occurred: ${err ${err. .message message}` }`) ); return process process. .exit exit( (0); } return console console. .log log( ('Message sent: %s', %s', info info. .messageId messageId) );
});
I ran the script scr ipt and it was sent successfully s uccessfully..
$ node examples/015/send-email.js examples/015/send-email.js "OHHHH NOOOO" "The game I was trying to build has missing information: Game name, Primary Colour Theme" Message sent: e.com>
Summary Now we know how easy it is to send an email via SMTP by executing a Node script. We We pass two arguments and let the script deal with creating the transporter via the
nodemailer
package and constructing the message object, and then sending
the email. As a side note, if you want to send an email quickly and easily, create a script that takes three simple arguments: recipient email address, subject and body. It’s It’s quicker than opening up a mail client and going through the steps of constructing an email.
016 - SMS So, we have done email, why don’t we expand our horizons by doing a similar thing with SMS. Let’s create a quick script that will send an SMS to our spouse informing him/her what the plan is tonight for food. I LOVE FOOD! A great service known as Twilio Twilio allows you to buy a phone number and to programmatically send messages - among other things. Lucky for you, they offer a free trial account which allows you to get a free phone number to send messages from. This applies at the time of writing of course. The catch is that the trial account will only allow you to send messages to verified phone numbers - meaning numbers you’ve added to your account and confirmed with the verification code they send to that number. So you’ve got two choices: use a free account to play around with this functionality, functionality, or pay for a full account and not be limited. To get started, go to http://www.twilio.com , and look for the sign up button. Once signed up, click on ‘Get Started’, and then click on ‘Get a number ’ in the following page. Finally, from your project’s dashboard, click on ‘Show API Credentials’ to see your ‘ACCOUNT SID’ and ‘AUTH TOKEN’. Now you’re you’ re all set up and ready to code. Just remember that until you add verified numbers, you will only be able to message yourself, as that is the only verified phone number you have at the start. If you have run the
npm run init
script initially, a
config.json
file has been pre-
created for you to hold the Twilio Twilio credentials and phone numbers. numbers . Replace the ’X’s ’X’s with your own generated credentials, cr edentials, your new Twilio phone number, and your spouse’s phone number (or the phone number you want to send messages to).
examples/016/config.json { "TWILIO_SID": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX", "TWILIO_SID": "TWILIO_AUTH_TOKEN": "TWILIO_AUTH_TOKEN" : "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX", "TWILIO_PHONE_NUMBER": "TWILIO_PHONE_NUMBER" : "+44XXXXXXXXXX" "+44XXXXXXXXXX", , "MY_SPOUSE_NUMBER": "MY_SPOUSE_NUMBER" : "+44XXXXXXXXXX" }
examples/016/sms-spouse.js
First thing we want to get is the details from the input using
readline-sync
and use the
twilio
config.json
file. We We will read
library to send SMS messages.
const config = require require( ('./config' './config') ); const readLineSync = require require( ('readline-sync' 'readline-sync') ); const Twilio = require require( ('twilio' 'twilio') );
Extract the values from the JSON J SON file, then make an instance of Twilio. We We create a constant so that if the user doesn’t choose a food choice we can exit.
const NO_CHOICE_MADE = -1 -1; ; const {
}
TWILIO_SID, TWILIO_SID, TWILIO_AUTH_TOKEN, TWILIO_AUTH_TOKEN , TWILIO_PHONE_NUMBER, TWILIO_PHONE_NUMBER, MY_SPOUSE_NUMBER = config config; ;
const client = new Twilio Twilio(TWILIO_SID (TWILIO_SID, , TWILIO_AUTH_TOK TWILIO_AUTH_TOKEN) EN); ;
Mmmmm. We We have some choices that are making me salivate already. Cooking myself, hmm, maybe takeout tonight. const foodChoices = [
'spag bowl 2nite', 2nite', 'chinese takeaway 2nite', 2nite', 'pie n mash 2nite', 2nite', 'mushroom risotto', risotto', 'pizza and fries', fries', '2 recover from my lunch, no food plz!', plz!', '2 cook 2nite' ];
Using 0
keyInSelect
we can allow the user to choose from the array. array. If they choose
we will will abort sending a SMS message.
const index = readLineSync readLineSync. .keyInSelect keyInSelect(foodChoices (foodChoices, , 'What would you like for dinner?') dinner?'); if (index === NO_CHOICE_MADE NO_CHOICE_MADE) ) {
process. process .exit exit( (0); }
So to conform confor m to Twilio’ Twilio’ss expected SMS object, we create like so: const smsMessage = {
body: `Hi Bub, I'd like ${foodChoices[index] body: ${foodChoices[index]}` }`, , from: from : TWILIO_PHONE_NU TWILIO_PHONE_NUMBER MBER, , to: to : MY_SPOUSE_NUMB MY_SPOUSE_NUMBER ER }; console. console .log log( (`sending message: ${smsMessage ${smsMessage. .body body}` }`) );
And we send it using the create method: // Send the text message.
client.messages client. messages. .create create(smsMessage) (smsMessage) .then then(( (({ { sid }) => { console. console .log log( ('SMS sent. Id:', Id:', sid) sid); ;
}) .catch catch((error) ((error) => { console. console .error error( ('Error sending Twilio message', message', error) error); ; });
Here is the script in action. $ node examples/016/sms-spouse.js examples/016/sms-spouse.js [1] [2] [3] [4] [5] [6] [7] [0]
spag bowl 2nite chinese takeaway 2nite pie n mash 2nite mushroom risotto pizza and fries 2 recover from my lunch, no food plz! 2 cook 2nite CANCEL
What do you want for dinner? [1...7 / 0]: 2 SMS sent. Id: SM30b413a3ce42410e873c4c5875d1d3ca SM30b413a3ce42410e873c4c5875d1d3ca
Good it sent. I really hope the chips are crispy this time.
Summary Yes, we have WhatsApp, but unfortunately they don’t have an API. For now we will settle for second best. The benefit is that if your spouse doesn’t have an Internet connection, there is more chance she will get your message. Yes, Yes, I have experienced living in the middle of nowhere. Come on WhatsApp, you can do it.
017 - CSV What if we wanted to batch-create ten games rather than creating them one-byone? There have been many times in my career where I have used CSV files for the use of batch processing. I find a lot of spreadsheets used by other teams that have valuable data in them that I can use to my advantage. Especially in my last company, company, there was a huge schedule of promotions pr omotions to go live with all the information I needed to create them. I deployed all the promotions for a whole month in one go. A stakeholder contacted me and as ked if the promotion was done. I said, s aid, “Yes “Yes mate, 22 days ago”. Here I will demonstrate a simplified example of how to process records one-byone from a CSV file. We begin by ensuring that our CSV file is consistently formatted and has two columns: the name of the game, and the game template it’s it’s based on.
Please note: If you have data stored in an application like Microsoft Excel or Google Sheets, you can export ex port that file to CSV directly - you don’t need to recreate it manually. manually.
examples/017/game-releases.csv Check Mate Chess,chess, Deluxe Backgammon,ba Backgammon,backgammon, ckgammon, Chaps of Checkers,draug Checkers,draughts, hts, Wild East Poker,poker, Kent World Poker,poker, Drake Draughts,draug Draughts,draughts, hts, Golden Backgammon,ba Backgammon,backgammon, ckgammon, BluffMe Poker,poker, Challenge of Chess,chess, SpinMe Slots,slots,
We want to write a script sc ript that reads the contents of this CSV file, parses it,
transforms it, and then pipes to standard output. But first we should explain the concept of streams in Node.
Streams As mentioned earlier in the book, streams are a pattern that makes a conversion of huge operations and breaks it down into manageable chunks. Kids, don’t eat a birthday cake whole, eat it in chunks. It’s It’s important to appreciate streams when dealing with large files. Using ten records from a CSV like what will be used in our example is not much of an issue, but if we were dealing with millions of records, this is where streams flex their muscles. If our CSV file did have millions of records, and we tried to load the entire contents of it by using the
fs.readFile
function, then we would have to load all of
it into memory at once. That’s quite a load, and it would be a bad idea for many reasons: bigger load on the server, bigger download for the user, longer loading time, and in the context of cake, a crippling stomach ache. There are many flavours of streams: 1. Readable streams - they act as the source. 2. Writable streams - they they act as the destination. 3. Duplex - both readable and writable. writable. 4. Transform - a special kind of Duplex Duplex stream. Input goes into the Transform Transform stream, and is returned modified or unchanged. We will use a Transform stream when parsing our CSV file. So, starting with our required modules. We We have two new additions: add itions: 1.
csv-parse
is responsible for the parsing of a CSV file: it can convert CSV
into JSON at a rate of around 90,000 rows per second.
2.
stream-transform
is used to provide a callback callback for each row sequentially for
convenience.
examples/017/deploy-games.js require('colors' require( 'colors') ); const path = require require( ('path' 'path') ); const { stdout } = require require( ('process' 'process') ); const parse = require require( ('csv-parse' 'csv-parse') ); const transform = require require( ('stream-transform' 'stream-transform') ); const fs = require require( ('fs' 'fs') );
Let’s Let’s set up our parser, and inform it on how our data is separated. In our case, it’s it’s separated by commas. Now you’ll see instead of reading in the whole file into memory with
fs.readFile
, we are setting up a read stream using the the native
fs
module, and declaring a simple iterator variable. The delay time is used to slow down the process for demonstration purposes. const DELAY_TIME = 500 500; ; const CSV_FILE = 'game-releases.csv' 'game-releases.csv'; ; const parser = parse parse( ({ delimiter delimiter: : ',' }); const gameReleasesPa gameReleasesPath th = path path. .join join(__dirname (__dirname, , CSV_FILE) CSV_FILE); ; const input = fs fs. .createReadStream createReadStream(gameReleasesPath) (gameReleasesPath); ; let iterator = 1;
For every record chunk we read, we want to do something. So we create a callback function to run on each iteration. const processRecord = (record (record, , callback) => { const [game [game, , template] = record record; ; let message = `Deploying game ${iterator ${iterator} } '${game '${game}' }' with template: '${template '${template}'` }'`; ;
message = (iterator % 2 === 0) ? message message. .bgGreen bgGreen : : message message. .bgBlue bgBlue; ; iterator += 1; setTimeout(() setTimeout (() => { // build game here
callback( callback (null, `${ `${message message}\n` }\n`) ); }, DELAY_TIME); }, DELAY_TIME) ; };
We extract the game and template from fr om the
record
array, array, then construct a
message to feed back to the standard output. We use
colors
to differentiate each
line from the next. This is where we would add the code to deal with actually building a game, but we’ll leave that for a future lesson. As a side note, you would remove the
setTimeout
from this script to speed things up of course.
const transformer = transform transform(processRecord) (processRecord); ;
The callback
processRecord
is passed to the transform function. Finally, Finally, below we
pipe the output of the parser to the transformer, which in turn passes its output to standard output.
pipe The
pipe
function reads data from a readable stream as it it becomes
available and writes it to a destination writing stream. We We are doing three
pipes in our example below. First it reads in a line from f rom the CSV file, sends to the parser. The parser parses it and then passes it to the transformer, the transformer forwards to standard output as we can see below when we run the script. input .pipe pipe(parser) (parser) .pipe pipe(transformer) (transformer) .pipe pipe(stdout) (stdout); ;
This is how the output looks. You can see it’s it’s processing each line of the file: $ node examples/017/deploy-games.js examples/017/deploy-games.js Deploying game 1 'Check Mate Chess' with template: 'chess' Deploying game 2 'Deluxe Backgammon' with template: 'backgammon' Deploying game 3 'Chaps of Checkers' with template: 'draughts' Deploying game 4 'Wild East Poker' with template: 'poker' Deploying game 5 'Kent World Poker' with template: 'poker' Deploying game 6 'Drake Draughts' with template: 'draughts' Deploying game 7 'Golden Backgammon' with template: 'backgammon' Deploying game 8 'BluffMe Poker' with template: 'poker' Deploying game 9 'Challenge of Chess' with template: 'chess' Deploying game 10 'SpinMe Slots' with template: 'slots'
Summary CSV files are a powerful way to batch specific processes. There is always a situation at work where they can be used to speed things up. Give it a go!
018 - Shorten URL Social Media is paramount to the success of any company. company. As a games studio for example, you would have to market your game and provide links. These links can get quite long and take up space which is a problem when you have the constraints of character limits.
Link Shorteners There are services such as Bitly & TinyURL, which are referred to as ‘link shorteners’. They take a long URL and shrink it into a smaller equivalent. Take this URL from my blog as an example.
Long URL http://smks.co.uk/travel/bali-2 http://smks.co. uk/travel/bali-2017/bali-2017-p 017/bali-2017-part-12/ art-12/
After running it through a service such as Bitly.
Short URL http://bit.ly/2jklAQb
The benefits are more than just fewer characters. If you sign up to one of the services, you should be able to track when links have been shared and clicked on. Let’s walk through an example of passing a URL as an argument and receiving back the URL in the shortened form. for m.
Please note: Before you proceed with this example ensure you have: created a bit.ly account, created a generic access token, and copied this access token into examples/018/config.json
.
examples/018/config.json { "BITLY_TOKEN": "BITLY_TOKEN" : "XXXXXXXXXXXXX "XXXXXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXXXXXX" XXXXXXXXXXXX" }
examples/018/shorten-url.js
As always, we require our dependencies at the top. We load in the
bitly
package
installed via npm, as well as the configuration that contains our access token. const Bitly = require require( ('bitly' 'bitly') ); const { BITLY_TOKEN } = require require( ('./config' './config') );
Now we have the token and dependency, we can instantiate Bitly and pass in the token. When making a request req uest to bit.ly, bit.ly, based on the documentation, we should expect a HTTP status code response of
200
to confirm everything went okay and
that the URL has been shortened. const STATUS_CODE_OK = 200 200; ; const bitly = new Bitly Bitly(BITLY_TOKEN) (BITLY_TOKEN); ;
Because we want to pass a URL as an argument, as done before we grab the URL string passed by the user. const args = process process. .argv argv. .slice slice( (2);
const [urlToShorten] = args args; ;
So
urlToShorten
will be something like ‘ http://www.opencanvas.co.uk http://www.opencanvas.co.uk’. ’. Just to
make sure it’s actually a URL before making a request to Bitly, we do some validation to save the overheads of a HTTP Request if it’s it’s invalid. To check it’s it’s a valid URL, we use a regular expression. const expression = /https /https? ?:\/\/( \/\/(www www\.)?[-a-zA-Z0 \.)?[-a-zA-Z0-9@:%._+~#=]{2, -9@:%._+~#=]{2,256}\.[a-z]{2,6 256}\.[a-z]{2,6}\b([-a-zA-Z0-9 }\b([-a-zA-Z0-9@:%_+.~#?&//=]* @:%_+.~#?&//=]*) )/gi /gi; ; const regex = new RegExp RegExp(expression) (expression); ; if (urlToShorten === undefined || urlToShorten urlToShorten. .match match(regex) (regex) === null) {
console.log console. log( ("Please pass a string in URL form, e.g. 'http://www.opencanvas.co.uk'" 'http://www.opencanvas.co.uk'") ); process. process .exit exit( (0); }
If the process is still running at this point, it means a valid URL was passed to our script. Next, we call the
shorten
method, passing the URL as an argument.
This method returns a Promise, which once fulfilled - if the request was successful - will contain our shortened URL. This new URL will be contained in response.data.url
.
bitly.shorten bitly. shorten(urlToShorten) (urlToShorten) .then then((response) ((response) => { const statusCode = response response. .status_code status_code; ; const statusText = response response. .status_txt status_txt; ; if (statusCode !== STATUS_CODE_OK) { console. console .error error( ('Something went wrong:', wrong:', statusText) statusText); ; } else { console. console .log log( (`Shortened URL is: ${response ${response. .data data. .url url}` }`) ); } }) .catch catch( (console console. .error error) );
And here is the output. $ node examples/018/ examples/018/shorten-url.js shorten-url.js "http://smks. "http://smks.co.uk/travel/ba co.uk/travel/bali-2017/bali-20 li-2017/bali-2017-part-12/" 17-part-12/" Shortened URL is: http://bit.ly/2 http://bit.ly/2jklAQb jklAQb
Summary URL Shorteners are a great way to track clicks of links, and a quick way to retrieve a shortened URL when you need to paste one on a social media platform.
019 - Minimist Up until now, now, we have been passing arguments to our scripts like so: $ node my-script.js 'My Template' 'My Ticket' 'My Game' // run script
But it isn’t very expressive. Like I may want to pass values without having to concern myself with how they are ordered. I would want something like this. $ node my-script.js --game='My Game' --ticket='My Ticket' --template='My --template='My Template' // run script
This way, regardless of the order these values get passed, I know that the correct value is being picked up. We can do this with a module called module will parse arguments from the
process.argv
minimist
. This
array and transform it in to to an
object. This allows you easy access to the arguments, as they will now be available as key-value pairs inside the object. We’ll stick with the convention above
--option=value
examples/019/minimist.js
We pass the
argv
array directly to
minimist
const argv = require require( ('minimist' 'minimist')( )(process process. .argv argv. .slice slice( (2)) )); ;
.
.
That’s That’s it. We have the object we need. Passing the following arguments and options, we can inspect the object of
argv
once it has run through
minimist
.
$ node examples/019/ examples/019/minimist.js minimist.js apple bananas cherries --name='My Game' --ticket='My Ticket' --template='My Template'
So what does our
argv
variable look like?
{ _ _: : [ 'apple' 'apple', , 'bananas' 'bananas', , 'cherries' 'cherries' ] ], , name: name : 'My Game', Game', ticket: ticket : 'My Ticket', Ticket', template: template : 'My Template' }
As you can see it pushes any arguments into an ordered array under an underscore key, but if you provide options, it will create key value pairs. Now on with the rest of the script. const readLineSync = require require( ('readline-sync' 'readline-sync') ); const NO_CHOICE_MADE = -1 -1; ; let { name name, , template template, , ticket } = argv argv; ; const templates = [ ['pick-of-three' 'pick-of-three', , 'tic-tac-toe' 'tic-tac-toe', , 'spin-the-wheel' 'spin-the-wheel'] ];
The good thing about this script is that it will have the possibility p ossibility to pass the values directly through instead of prompting us. But… if the values are not found, only then will the script become interactive and prompt the user for the value it needs. You can see s ee we can assign name, template and ticket right away, and below that we have an array of supported templates. If the template chosen is not in this list then we are going to have problems. “Wee should not give up and we should not allow the problem to defeat us.” “W – A. P. P. J. Abdul Kalam We won’t let it happen, Abdul.
If the name wasn’t passed as an option, then we simply prompt the user to give it to us. We’re We’re not going to all this trouble for nothing, you know! if (name === undefined) {
name = readLineSync readLineSync. .question question( ('What is the name of the new game? ', ', { limit: limit : input => input input. .trim trim(). ().length length > 0, limitMessage: limitMessage : 'The game must have a name' }); }
Now we are going to check if the user even bothered to pass the template. But it doesn’t stop there. The array of templates we declared earlier has to match our entered choice too, otherwise we tell the user to make a choice from the ones that do exist. if (template === undefined || !templates templates. .includes includes(template)) (template)) { const templateIndex = readLineSync readLineSync. .keyInSelect keyInSelect(templates (templates, , 'Choose your template') template'); if (templateIndex === NO_CHOICE_MADE NO_CHOICE_MADE) ) {
console.log console. log( ('No template chosen. Stopping execution.') execution.'); process. process .exit exit( (0); } template = templates[templ templates[templateIndex] ateIndex]; ; }
Now we are going to check if the ticket was passed, and ensure – due to business rules – that the ticket begins with ‘GS-’. Once all 3 values have been acquired and they satisfy our criteria, we can proceed to building the game. if (ticket === undefined || ticket ticket. .indexOf indexOf( ('GS-' 'GS-')) )) {
ticket = `GS-${ `GS-${readLineSync readLineSync. .question question( ('Enter ticket number: GS-', GS-', { limit: limit : input => input input. .trim trim(). ().length length > 0, limitMessage: limitMessage : 'Cannot continue without a ticket number' })}` }`; ; } console. console .log log( (`Creating game '${name '${name}' }' with template '${template '${template}' }' on branch '${ticket '${ticket}'` }'`) );
Let’s Let’s see what the output ou tput is. $ node examples/019/minimist.js --name="chess" --template="pick-of-three" --template="pick-of-three" --ticket="GS-100" Creating game 'chess' with template 'pick-of-three' on branch 'GS-100'
Now let’s let’s try it by not n ot passing any options at all:
$ node examples/019/minimist.js What is the name of the new game? choose-your-prize [1] [2] [3] [0]
pick-of-three tic-tac-toe spin-the-wheel CANCEL
Choose your template [1, 2, 3, 0]: 1 Enter ticket number: GS-346 Creating game 'choose-your-prize' with template 'pick-of-three' on branch 'GS-346'
Summary Prompting users is fine in some cases, but sometimes, like when you want to let a Cron job handle the process, we need to pass arguments to the script so it can be run automatically. automatically. Minimalist allows us to do that.
020 - Build Now we are going to work on a slightly bigger example to demonstrate the build of a game. This game would’ve been implemented already and released as a specific version. This example we will be using is a slightly tweaked version of the original Firefox game that can be found here https://developer.mozilla.org/enUS/docs/Games/Tutorials/2D_Breakout_game_pure_JavaScript.. US/docs/Games/Tutorials/2D_Breakout_game_pure_JavaScript The difference with this version is that you can create a configuration file that can alter the visuals of the game. The values will be injected into the game. This example will be based around two configurable values: a primary colour, and a secondary colour. The game only has two colours, and they should compliment each other other.
examples/020/template/game.json { "primaryColor": "#fff" "primaryColor": "#fff", , "secondaryColor": "secondaryColor" : "#000" }
The primary colour will be used to determine the colour of the game objects, and the secondary colour will paint the background. So we are empowering the JSON file to configure the game’s game’s ‘skin’. └─020 ├─core | └─game-1.0.0.js ├─releases ├─template └─new-reskin.js
The root of the
020
directory contains three three directories and the script we will run
to build a new release. The first directory
will contain the bundled files of
core
the game using iterative versions. Using versioning will prevent backward compatibility issues with existing game releases. The
releases
directory will contain each of the reskins we make of the game
template. It’s It’s as simple as cloning the
template
directory, directory, and changing the name,
and updating the JSON file so it changes into its own unique version.
Please note: This example uses only two values, but later on we will use a more advanced implementation that will contain images, fonts and custom styles. There is nothing stopping you building upon this example. Before we run the script, we want to run a simple server. To To do this, you need to run this in the root of
nobot-examples
repository in a separate CLI. Your Your public IP
address shown below will most likely be different. $ npm run gameServe > http-server examples/020 Starting up http-server, serving examples/020 Available on: http://192.168.1.68:8080 http://127.0.0.1:8080
Great, we now have a server running and pointing to the root of the
020
directory. directory. Now we can walk through the script. At the top of the script we require the packages needed. 1.
colors
2.
minimist
to be used to signify success or failure. from the last example to make it easier to pass arguments to our
script and to parse them optionally. optionally. Pass input without being prompted to enter.
3.
path
to construct paths to to the template and the destination of the new game. game.
4.
readline
5.
fs-extra
6.
opn
-sync to prompt user for information if it’s missing. so we can copy and paste our game directories.
is a library that that is cross platform and will open up our game in a
browser upon completion.
examples/020/new-reskin.js require('colors' require( 'colors') ); const argv = require require( ('minimist' 'minimist')( )(process process. .argv argv. .slice slice( (2)) )); ; const path = require require( ('path' 'path') ); const readLineSync = require require( ('readline-sync' 'readline-sync') ); const fse = require require( ('fs-extra' 'fs-extra') ); const open = require require( ('opn' 'opn') );
As demonstrated in the previous example with the minimist library, library, we can pass in the key value pair options directly to the script, therefore avoiding the need to prompt the user for each required value. This is what we are obtaining here, and our first check is to see if the game name has been passed in. If it hasn’t then we prompt the user for it, applying some simple validation. const GAME_JSON_FILE GAME_JSON_FILENAME NAME = 'game.json' 'game.json'; ; let { gameName gameName, , gamePrimaryColo gamePrimaryColor r, gameSecondaryC gameSecondaryColor olor } = argv argv; ; if (gameName === undefined) {
gameName = readLineSync readLineSync. .question question( ('What is the name of the new reskin? ', ', { limit: limit : input => input input. .trim trim(). ().length length > 0, limitMessage: limitMessage : 'The project has to have a name, try again' }); }
Because two of our values need to be Hex codes, we create a function that can do the check for both colours: the primary and the secondary. If the colour supplied by the user did not pass our validation, we prompt for the colour until it does. const confirmColorIn confirmColorInput put = (color (color, , colorType = 'primary' 'primary') ) => { const hexColorRegex = /^#([A-Fa-f0-9]{6}|[A-Fa-f0-9]{3})$/; if ( (hexColorRegex hexColorRegex. .test test(color)) (color)) { return color color; ;
} return readLineSync readLineSync. .question question( (`Enter a Hex Code for the game ${colorType ${colorType} } color `, `, {
limit: hexColorRegex limit: hexColorRegex, , limitMessage: limitMessage : 'Enter a valid hex code: #efefef' }); };
We use the same function to obtain both the primary and secondary colours. gamePrimaryColor = confirmColorInput(gamePrimaryColor confirmColorInput(gamePrimaryColor, , 'primary' 'primary') ); gameSecondaryColor = confirmColorInput confirmColorInput(gameSecondaryColor (gameSecondaryColor, , 'secondary' 'secondary') );
In the next block of code we are printing to standard output to confirm the values that will be used in the process of building the game. The statements that follow are preparing the paths to the relevant files and directories. The
src
will point to
the template directory. directory. The
destination
will point to a new directory under releases. The configuration configuration file
that will have its values updated will reside under this new game directory we are creating. And finally, to preview our new game, we construct the URL using the path to the local server we booted up earlier on. console.log console. log( (`Creating a new reskin '${gameName '${gameName}' }' with skin color: Primary: '${gamePrimaryColor '${gamePrimaryColor}' }' Secondary: '${gameSecondaryColor '${ gameSecondaryColor}'` }'`) ); const src = path path. .join join(__dirname (__dirname, , 'template' 'template') ); const destination = path path. .join join(__dirname (__dirname, , 'releases' 'releases', , gameName) gameName); ; const configurationF configurationFilePath ilePath = path path. .join join(destination (destination, , GAME_JSON_FILEN GAME_JSON_FILENAME) AME); ; const projectToOpen = path path. .join join( ('http://localhost:8080' 'http://localhost:8080', , 'releases' 'releases', , gameName gameName, , 'index.html' 'index.html') );
In the following code:
1. We copy the template template files to the releases directory. directory. 2. After this is created, we read the JSON of the original template. template. 3. With the new configuration object, we override the existing primary and secondary colours provided by the user. 4. We rewrite the JSON JSON file so it has the new values. 5. When the JSON file file has been updated, we we ask the user if they would like like to open the new game in a browser. We can only use the copy function below using the Please note: We package. I have named it as native
fs
fse
fs-extra
to differentiate differentiate between this version and the
package.
fse.copy fse. copy(src (src, , destination) .then then(() (() => { console. console .log log( (`Successfully created ${destination ${destination}` }`. .green green) ); return fse fse. .readJson readJson(configurationFilePath) (configurationFilePath); ; }) .then then((config) ((config) => { const newConfig = config config; ; newConfig. newConfig .primaryColor = gamePrimaryCol gamePrimaryColor or; ; newConfig. newConfig .secondaryColor = gameSecondaryCo gameSecondaryColor lor; ; return fse fse. .writeJson writeJson(configurationFilePath (configurationFilePath, , newConfig) newConfig); ; }) .then then(() (() => { console. console .log log( (`Updated configuration file ${configurationFilePath ${configurationFilePath}` }`. .green green) ); openGameIfAgreed(projectToOpen) openGameIfAgreed (projectToOpen); ; }) .catch catch( (console console. .error error) );
Below is the function that gets invoked when the copying has completed. It will then prompt the user to see if they would like to open up the game in the browser. const openGameIfAgre openGameIfAgreed ed = (fileToOpen) => { const isOpeningGame = readLineSync readLineSync. .keyInYN keyInYN( ('Would you like to open the game? ') '); if (isOpeningGame) {
open(fileToOpen) open (fileToOpen); ; } };
Here is the entire script.
examples/020/new-reskin.js require('colors' require( 'colors') ); const argv = require require( ('minimist' 'minimist')( )(process process. .argv argv. .slice slice( (2)) )); ; const path = require require( ('path' 'path') ); const readLineSync = require require( ('readline-sync' 'readline-sync') ); const fse = require require( ('fs-extra' 'fs-extra') ); const open = require require( ('opn' 'opn') ); const GAME_JSON_FILE GAME_JSON_FILENAME NAME = 'game.json' 'game.json'; ; let { gameName gameName, , gamePrimaryColo gamePrimaryColor r, gameSecondaryC gameSecondaryColor olor } = argv argv; ; if (gameName === undefined) {
gameName = readLineSync readLineSync. .question question( ('What is the name of the new reskin? ', ', { limit: limit : input => input input. .trim trim(). ().length length > 0, limitMessage: limitMessage : 'The project has to have a name, try again' }); } const confirmColorIn confirmColorInput put = (color (color, , colorType = 'primary' 'primary') ) => { const hexColorRegex = /^#([A-Fa-f0-9]{6}|[A-Fa-f0-9]{3})$/; if ( (hexColorRegex hexColorRegex. .test test(color)) (color)) { return color color; ;
} return readLineSync readLineSync. .question question( (`Enter a Hex Code for the game ${colorType ${colorType} } color `, `, {
limit: hexColorRegex limit: hexColorRegex, , limitMessage: limitMessage : 'Enter a valid hex code: #efefef' }); }; gamePrimaryColor = confirmColorInput(gamePrimaryColor confirmColorInput(gamePrimaryColor, , 'primary' 'primary') ); gameSecondaryColor = confirmColorInput confirmColorInput(gameSecondaryColor (gameSecondaryColor, , 'secondary' 'secondary') ); console.log console. log( (`Creating a new reskin '${gameName '${gameName}' }' with skin color: Primary: '${gamePrimaryColor '${gamePrimaryColor}' }' Secondary: '${gameSecondaryColor '${ gameSecondaryColor}'` }'`) ); const src = path path. .join join(__dirname (__dirname, , 'template' 'template') ); const destination = path path. .join join(__dirname (__dirname, , 'releases' 'releases', , gameName) gameName); ; const configurationF configurationFilePath ilePath = path path. .join join(destination (destination, , GAME_JSON_FILEN GAME_JSON_FILENAME) AME); ; const projectToOpen = path path. .join join( ('http://localhost:8080' 'http://localhost:8080', , 'releases' 'releases', , gameName gameName, , 'index.html' 'index.html') );
fse.copy fse. copy(src (src, , destination) .then then(() (() => { console. console .log log( (`Successfully created ${destination ${destination}` }`. .green green) ); return fse fse. .readJson readJson(configurationFilePath) (configurationFilePath); ; }) .then then((config) ((config) => { const newConfig = config config; ; newConfig. newConfig .primaryColor = gamePrimaryCol gamePrimaryColor or; ; newConfig. newConfig .secondaryColor = gameSecondaryCo gameSecondaryColor lor; ; return fse fse. .writeJson writeJson(configurationFilePath (configurationFilePath, , newConfig) newConfig); ; }) .then then(() (() => { console. console .log log( (`Updated configuration file ${configurationFilePath ${configurationFilePath}` }`. .green green) ); openGameIfAgreed(projectToOpen) openGameIfAgreed (projectToOpen); ; }) .catch catch( (console console. .error error) );
const openGameIfAgre openGameIfAgreed ed = (fileToOpen) => { const isOpeningGame = readLineSync readLineSync. .keyInYN keyInYN( ('Would you like to open the game? ') '); if (isOpeningGame) {
open(fileToOpen) open (fileToOpen); ; } };
Let’s see it in action. $ node examples/020/new-reskin.js examples/020/new-reskin.js What is the name of the new reskin? shaun-the-sheep Enter a Hex Code for the game primary color #fff Enter a Hex Code for the game secondary color #000 Creating a new reskin 'shaun-the-sheep' with skin color: Primary: '#fff' Secondary: '#000' Successfully created /Users/shaun/Wo /Users/shaun/Workspace/nobot-e rkspace/nobot-examples/example xamples/examples/020/releases/ s/020/releases/shaun-the-sheep shaun-the-sheep Updated configuration file /Users/shaun/W /Users/shaun/Workspace/nobotorkspace/nobot-examples/exampl examples/examples/020/releases es/020/releases/shaun-the-shee /shaun-the-sheep/game.json p/game.json Would you like to open the game? [y/n]: y (opens game in browser)
Summary Using a simple example of a small game, we have created a script that can build a new game from a template, using custom colours provided by the user, we can create many clones of the game and use different colours for each. A dumbed down version of what is to come in part 2!
Part 2 - Build Tool Excellent, you’ve now warmed up your fingers, but here is the real deal; part 2. Before you proceed, carefully follow the instructions below so you’re ready to rock. Make sure you have a GitHub account if you want to write these scripts from scratch. Alternatively you can browse the completed scripts sitting on the
master
branch. 1. Fork the repository under your own name. This can be done by clicking clicking the Fork button on the GitHub page here https://github.com/smks/nobot https://github.com/smks/nobot.. So rather than the repository being under my own name ‘smks’ it will will be under yours instead. 2. Do another another fork for the repository https://github.com/smks/nobot-website https://github.com/smks/nobot-website.. 3. Do another another fork for the repository https://github.com/smks/nobot-templaterock-paper-scissors.. rock-paper-scissors 4. Clone your forked repository
nobot
to your own machine (No need to clone
other two forked repositories). 5. Switch to branch 6. Run
npm install
develop
by doing
git checkout develop
in the root of the repository. repository.
7. Follow along with the the nobot implementation. 8. Happy coding!
.
Scenario You, a developer that works for Nobot Game Studios, make many new games on a regular basis. The business has now decided that making multiple versions of the same game is a more viable alternative. The idea is that the team builds a game that can be configurable and reskinnable, but retains the same mechanics. The first meeting that takes place involves the various teams involved: 1. Stakeholders and/or Product Owners - Responsible for the games and how they are delivered. 2. Content Editors - Will Will provide the witty text content (Title, (Title, subtitles, labels). 3. Designers - The creative geniuses who will design the assets for your game. 4. Developers - The poor souls who have to deal with all the the functionality of the game - and the many changes. 5. Quality Assurance - Will Will test and verify the initial game template and reskins. After many disagreements, back and forth suggestions and terrible jokes, you agree on the following flow. 1. The product owner, owner, after doing some heavily involved research, decides decides to create an original game called ‘Rock Paper Scissors’. Five different versions of this game will be created to show on the main games lobby. The product owner communicates the idea to the other teams, and says that he/she will create a ticket for each of the games that contains all of the necessary content (obtained from the content editors and designers). 2. The game designers start off by creating a design template that will speed
up their real-world implementations. The game designers get to work on the implementations for all five games, using the original template as the basis for their designs. When they are finished, they upload their assets to a CDN (Content Delivery Network) repository and provide the link to the assets for the live game in our project management software. E.g. http://cdn.opencanvas.co.uk/automatingwithnodejs/assets/rock-paperscissors/rock-paper-scissors-doodle 3. The developers get to work on the template template implementation, implementation, using the provided live links to the assets and making sure the template is configurable to the point that it can satisfy the requirements of all five game implementations. For this, it will require some team planning to ensure things go smoothly. smoothly. 4. The game gets released, and is tested firstly firstly by QA and then anyone else who wants to join in on the fun. 5. Rinse and repeat for new games.
Development Meeting You HAVE HAVE to plan a meeting with your fellow team. Sit down in a meeting room around a whiteboard, pick up a black or blue pen and each of you write what you think happens from A to Z. We did this at my old company, and although there was a lot of digression, we managed to come to an agreement after a few months. Only joking, we never agree on anything. For this we did though. Here is a mock example of something that would be conjured up in a meeting for the new Rock Paper Scissors game.
Required Tech 1. Source Control - Git. 2. Vanilla JS - AKA - JavaScript (Don’t always need a framework, kids). kids). 3. ESLint to ensure code quality and standards. 4. Babel so we can write for the future (ES6, 7 and beyond). 5. Agile Methodology to Software Development along with an issue management tool (In house project management tool called Nira also has an API). 6. Brunch - Used to watch, build, compile and concatenate game production files (Like Webpack, Gulp). A lightweight lightweight simple implementation used for this book. 7. Node.js to to automate our delivery process.
Required Repositories The team has agreed to create the repositories and give them the detailed responsibilities below. below.
Nobot Content Delivery Network (CDN)
A place where designers upload assets for a game. These assets will be pulled into the game. Rock Paper Scissors will have five different skins, each of which having five collections of assets. The location of these assets is added to the Nira ticket. This repository will be created and handed over to the designers to manage and take ownership of. They will need to follow an approach that is consistent. The repository below shows the five design implementations exported from the designers. GitHub Repository
https://github.com/smks/nobot-cdn Production Link CDN
http://cdn.opencanvas.co.uk/automatingwithnodejs Directory structure └── assets └── rock-paper-scis rock-paper-scissors sors
├── │ │ │ │ │ │ ├── │ │ │ │ │ │ ├── │ │ │ │ │ │ ├── │ │ │ │ │ │ └──
fire-water-earth-cute fire-water-ear th-cute ├── background.png ├── banner.jpg ├── paper.png ├── rock.png ├── scissors.png └── theme.css fire-water-earth-fantasy fire-water-ear th-fantasy ├── background.png ├── banner.jpg ├── paper.png ├── rock.png ├── scissors.png └── theme.css fire-water-earth-retro fire-water-ear th-retro ├── background.png ├── banner.jpg ├── paper.png ├── rock.png ├── scissors.png └── theme.css rock-paper-scissors-doodle rock-paper-sci ssors-doodle ├── background.png ├── banner.jpg ├── paper.png ├── rock.png ├── scissors.png └── theme.css rock-paper-scissors-modern rock-paper-sci ssors-modern ├── background.png ├── banner.jpg ├── paper.png ├── rock.png ├── scissors.png └── theme.css
So each design implementation has the same asset names, and a
theme.css
script
which applies the styling to our game. In our development implementation we would point to the correct directory to load in the assets. Nobot Website
This is the website where the players can play our games. Our games, once built, will be deployed here. GitHub Repository
https://github.com/smks/nobot-website
Production Link Website
http://ngs.opencanvas.co.uk Directory structure │ .gitignore │ bulma.css │ index.php │ main.css │ main.js │ README.md │ run.bat │ run.sh │ update-site.sh ├───core │ .gitkeep │ rock-paper-scissors.1.0.1.css │ rock-paper-scissors.1.0.1.js │ rock-paper-scissors.1.1.0.css │ rock-paper-scissors.1.1.0.js └───releases │ .gitkeep ├───fire-water-earth-cute │ game.json │ index.html ├───fire-water-earth-fantasy │ game.json │ index.html ├───fire-water-earth-retro │ game.json │ index.html ├───rock-paper-scissors-doodle │ game.json │ index.html └───rock-paper-scissors-modern game.json index.html
Under the
core
directory is where we will will release our transpiled and minified
game bundled files. As mentioned previously prev iously,, using versioning will allow us to make future implementations and keep backwards backward s compatibility to avoid breakages of existing game bundles. The
releases
directory will be where we deploy our various projects with their
unique identifiers. Each project release will contain an configuration file
game.json
index.html
file and a
with all the values captured from the Nira ticket.
Nobot Template Template - Rock Paper Scissors
The source code for the game template ‘rock-paper-scissors’. This is where we build our production code for our game. Remember you will be using your own forked version. GitHub Repository
https://github.com/smks/nobot-template-rock-paper-scissors The
index.html
will load in in the core CSS and JS bundle specified, and also provide
the HTML Markup that the JS bundle relies on to work correctly. Our games are Document Object Model (DOM) driven.
app/assets/index.html > <meta charset= charset="utf-8" "utf-8"> <meta name= name="viewport" "viewport" content= content="width=device-width" "width=device-width"> Game <script src= src="/core/rock-paper-scissors.1.1.0.js" "/core/rock-paper-scissors.1.1.0.js"> <script>require require( ('initialize' 'initialize') ); Loading... 
