Schedule Python scripts on Google Cloud Platform from scratch

A step-by-step guide from scratch to automate tasks using Google Cloud Platform

Create account

Assuming that you already have a Google account (if you don’t you will need to create one), go to the Google Cloud Platform homepage by clicking here or by clicking the link below:

Cloud Computing Services | Google Cloud

As a trial, there is $300 in free credits to spend on Google Cloud.

• Click theGet started for free button
• Select country and click on continue button
• Enter a phone number, verify it and click on continue button
• Select an individual account type and enter your information
• Click Start the free trial button

There will be no charges! Indeed, before being charged, the $300 free credits need to be spent — which will take a lot of time — and in case it happens, there will be warnings to tell you’re going to have to pay.

The Google Cloud Platform account is now created.

Create project

As you might have noticed, a project called My first project is already created when entering the account for the first time but we are going to create a new one for the exercise.

Click on My first project at the top left. A pop-up will show up where you can select existing project or create new one. Click on the New Project button.

• Name your project — here it is called scheduled-task
• Optional: under the Project Name text input, you can see a line indicating the project ID. If the proposed project ID suits you, leave it as it is otherwise, you can edit it. It will be used later
• Location: Leave No organisation
• Click Create button

Activate the project by clicking My first project and select the newly created project scheduled-task .

Billing needs to be enabled so that Compute Engine section can be used. As mentioned earlier, nothing will be charged yet.

Go to the navigation bar on the left, find Billing menu and click on it.

• Click on the Link a billing account button
• Click on the Create a billing account button
• Name the billing account
• Select country
• Click Continue button
• Check everything is ok and click Submit and enable billing button

Create a Virtual Machine

Now we are going to create a Virtual Machine that we will then start and stop for our needs. Find the Compute Engine section available on the side bar.

On Compute Engine page, click Enable button to activate this service on your account. Now, a VM can be created. Click on Create instance button.

• Name the instance as you wish — here let’s call it vm-scheduled
• Select a region depending on your location — here we use europe-west1 (Belgium) as the region and europe-west1-b as the zone
• Under Identity and API access, select Allow full access to all Cloud APIs
• Under Firewall tab, allow HTTP and HTTPS traffic
• Click Create button

The VM is now created and is running.

Configure Virtual Machine

Now we are going to configure the minimum requirements for the VM to be able to execute Python scripts. Click on SSH button on the running VM line, a terminal will pop-up.

A running VM does have a cost. Thus, we are going to keep it running only for the time needed to execute our scripts.

To do this, we are going to configure a script that will be automatically executed when the VM starts, and the last line of that script will be a request to shut down the VM.

Startup scripts are executed at the root of the VM, so we want to configure the environment at the root of the machine.

On the terminal, enter the following command and click enter:

sudo su

Now we are at root and we can setup the environment. Execute the following lines line by line.

sudo apt update
sudo apt install python3 python3-dev python3-venv
sudo apt-get install wget
wget https://bootstrap.pypa.io/get-pip.py
sudo python3 get-pip.py
python3 -m venv env

At this stage, Python 3 is installed:

python3 -V

The VM is now configured and ready to execute startup scripts.

Create startup script

We are going to create a HelloWorld python script that will be executed at machine startup. We need:

• the Python file containing the code to execute
• A .sh file in which we can write every instruction we want

touch helloworld.py
touch startupscript.sh

The .sh file will have to be executed, so we need to give it permissions:

chmod u+r+x startupscript.sh

Now it is necessary to use VIM editor to edit our files. I will not explain how to use VIM editor here but you will find everything needed by searching on the web or at the following address:

https://vim.rtorr.com/

At the end, the two files should contain the following instructions:

Let’s check it works:

• Exit the terminal window
• Stop the VM
• Click on the VM name
• Click on EDIT button

Find Custom metadata tab and add a line:

> KEY   = startup-script
> VALUE = /absolute_path_to_your_sh_file/startupscript.sh

Now the VM will execute the code each time it starts. Let’s check it by starting the machine. Click on SSH to open a terminal and execute the following command:

tail -n 100 /var/log/syslog

There will be a lot of information and you will find the hello world message somewhere.

You can now close the terminal and turn off the machine.

Now we need to set up the VM start and stop functions.

Enable Cloud Build API

To use the cloud functions, we need the Cloud Build API enabled.

• Find APIs & services in the left side menu and click on it
• Click on Enable APIs and Services button
• Search for Cloud Build API and click on it
• Click on the Enable button

Now, let’s create the start and stop functions.

Start Function

On the left side menu, find Cloud Functions and click on it.
Then click on the button Create Function

• Name the function — here we will name it start-vm
• Select region — the same you selected for the VM
• Trigger: keep HTTP, and select Allow unauthenticated invocations
• Click Save button
• Click Next button
• Select Python 3.7 as runtime
• As Entry point, write start_vm
• Copy/paste the following code in main.py and requirements.txt file

• Click Deploy button

ok !

Check it works:

• Click the Cloud Function name
• click on Testing button
• click on Test the function button

If everything OK, the output will be OK and if you go to your Compute Engine dashboard, you will see that the VM is starting, and of course will execute the startup code.

Note that the trigger of this Cloud function is set to HTTP. Thus, each request to the url associated to this function will execute it and make the VM start.

Get your start-vm trigger url

• Click on start-vm function
• Find Trigger tab and click on it

The same way, let’s create the stop function.

Stop function

Follow the steps above for the creation of the start function, but replace:

• name: stop-vm
• As Entry point, write stop_vm

The same way as we did for the start-vm function, you can test the stop-vm function and observe that the VM is stopping.

Get your stop-vm trigger url

• Click on stop-vm function
• Find Trigger tab and click on it

Now let’s create the event that will make the start function to execute automatically everyday.

Trigger Setup

We want our script to be executed, say everyday at 9am. To do this, we need to execute the start-vm cloud function everyday at 9am. It will start the VM which will execute the script.

On the left side menu, find Cloud Scheduler and click on it.

• Click on Schedule a job button
• Again, select the region
• Click Next button
• Name the trigger — here we name it startjob
• Optional: the description
• Frequency: O 9 * * 0–6
• Time zone: your time zone

The frequency is CRON format, you can learn how to use it by searching the web. There is a nice CRON generator at the following address:

https://crontab.guru/

For this example, we want the job to be executed everyday at 9am which is in CRON:

0 9 * * 0-6

• Click the Continue button

As a target, select HTTP and write in the URL text input the trigger address of the start-vm Cloud Function (see above how to find this address)

As a HTTP method, select GET (POST should work too)

• Click the Continue button
• Click the Create button

That’s it. Everyday at 9am, this scheduler will call the start-vm trigger url and the start-vm function will be executed. The VM will be launched and the script executed.

There is a last thing to do: Stop the virtual machine as soon as the script execution is finished.

Stop the virtual machine after script execution

To do this, we need to add a line to the startupscript.sh previously created.

Go to Cloud Functions service, select the stop-vm function and copy its trigger URL. Then:

• Go to Compute Engine
• start the VM
• Click on SSH to open terminal
• Install the Python package called requests

sudo su 
pip install requests

• Create a stop_instance.py file

touch stop_instance.py

And write the following code inside stop_instance.py:

• Open the startupscript.sh file and add the following line inside:

python3 /absolute_path_to_your_python_file/stop_instance.py

Thus the file startupscript.sh file should look like:

Finished

Now the VM will run and execute the script everyday at 9am before shutting down again.

I hope you like, if you have ideas of improvement or suggestions, please write it in comments or message me.