Install Puppeteer – An Excellent Learning Guide of Puppeteer Tutorial 4 & 5

by

Puppeteer is an open-source node js library that can be used for web scraping tools. It can also be used to perform test automation in web applications. Now-a-days, the usage of Puppeteer is getting increased rapidly in the automated software testing space. Basic knowledge of command line, Javascript, and HTML DOM structure is required to understand puppeteer tutorial. The entire tutorial is segregated into the below articles. 

Puppeteer Tutorial

Tosca Tutorial #1: Puppeteer Overview

Tosca Tutorial #2: Puppeteer Environment Variables

Tosca Tutorial #3: Puppeteer Web Scraping and Puppeteer Test Automation Overview

Tosca Tutorial #4: Install Puppeteer

Tosca Tutorial #5: Sample Puppeteer Project

In this Puppeteer Tutorial, we will learn about the steps to install Puppeteer with its dependencies such as install NodeJs, install editor for Puppeteer, etc. Also, after installation, we will create and execute one sample Puppeteer project.

Install Puppeteer

To start the development of Puppeteer scripts, we need to install and configure the below components – 

1. Install NodeJS

2. Install Editor

3. Install Puppeteer

Install NodeJS:

NodeJs is a free open source server environment which can be run in different platforms. It uses javascript in the server side. The Puppeteer is one kind of NodeJS application. So the first step of Puppeteer setup is to install the NodeJS framework. The NodeJS framework is available for multiple platforms, including Windows, Ubuntu, macOS, etc. In this context, we will work on the version for 64 bit Windows operating system. The steps to install NodeJS are –

Step1# Download NodeJS: Click here to navigate the NodeJS download link. Here, we will download the 64 bit windows installer (.mts). 

Puppeteer tutorial - Install NodeJs
Puppeteer tutorial – Install NodeJs

Step2# Install NodeJS: After completion of the download, we need to install NodeJs by double-clicking on the installer(.msi) file. During the installation, we need to proceed according to instructions.

Step3# Verify NodeJS: After the completion of the installation, we need to open the command prompt and enter the command as “node”. If the below details are appearing, the installation is correct. In case, if there is any error appears, that means installation is not correct.

Puppeteer tutorial - Verify NodeJs
Puppeteer tutorial – Verify NodeJs

Install Editor for Puppeteer:

An editor is nothing but a tool that helps us to write, compile and run our Puppeteer codes. There many tools are available which can be used as a java code editor which includes Visual Studio Code, Note Pad ++, Edit Plus, etc. Even we can write puppeteer code in the default “Note Pad” application as well. In this “Install Puppeteer” tutorial, we will use VSCode as it’s free and easily compatible with NodeJS Application. VSCode is nothing but one component of visual studio, which is available for free. Steps to install VSCode are – 

Step1# Downloadd VSCode: Click here to open the download link and download the desire version of VSCode Installer as per the operating system.

Step2# Install VSCode: Install VSCode from the installer file in the system just like any other software. During the installation, proceed with recommended setting only.

Step2# Verify VSCode: After the completion of the installation, open the application to check whether it’s installed correctly.

Puppeteer tutorial - Editor for Puppeteer
Puppeteer tutorial – Editor for Puppeteer

Install Puppeteer Packages:

From the version v1.7.0 of puppeteer, every release contains below two packages –

  • puppeteer-core package
  • puppeteer package

Both the versions of Puppeteer can be installed using the console commands. The commands to install Puppeteer are – 

Install Puppeteer-core Package: It’s a collection of Node JS library which is developed in Java. It has the ability to work on devtools protocol. The Chromium browser is not getting downloaded while installing the puppeteer-core package. The programmatic interface of Puppeteer completely drives the puppeteer-core library. Another important limitation is that the puppeteer-core features can’t be altered by changing any of the PUPPETEER_* environment variables. 

Installation Command: npm install puppeteer-core

Note: The Node JS tool need to be installed before installing the puppeteer-core package.

Install Puppeteer Product Package: Puppeteer is the complete product which is developed by Google to controls the Chrome browsers. Being the complete Puppeteer product package, the latest versions of chromium browser is getting downloaded during the installation. After that the installation is driven by puppeteer-core. It’s possible to customize the Puppeteer features by changing the PUPPETEER_* environment variables. 

Installation Command: npm install puppeteer

In this “Install Puppeteer” tutorial, we will work on the Puppeteer package installation as there are not many differences between these two versions.

Sample Puppeteer Project

The Puppeteer is compatible with both headful (non-headless) and headless chrome browsers. In case of headless, the browser activities are performed in the background i.e. the browser UI is not visible to us.  It make the thing (controlling the browser) simpler and easier in single step. It means, same thing(controlling the browsers) can be done with multiple complex steps.

The steps involved in setup sample Puppeteer project are shown below – 

Step1# Create a folder structure for Sample Puppeteer Project: Create a sample root directory with the name “SampleProject” in a pre-defined path. This root directory will be acted as a Sample Puppeteer Project. Next, after opening the command prompt, we need to navigate to this root directory.

Step2# Install Puppeteer: Using the below command, we can install the full package of Puppeteer in the root directory. This command basically downloads all the open-source NodeJS libraries in the sample project folder. The installation procedure takes some time based on the network speed. It will download approximately 350MBs of data. After installation, the node_modules folder, which contains different puppeteer components and package-lock.json file, will be created to the sample Pupeteer project root folder.

Puppeteer tutorial - Installation Log
Puppeteer tutorial – Installation Log

Step3# Create Sample Puppeteer Script: Now, we will write a sample puppeteer script that invokes the LambdaGeeks website, displays console messages after each step, and capture the screenshot. In this example, a headless chromium-browser will be invoked in the background. The sample Puppeteer Script will be – 

const puppeteer = require('puppeteer'); //include Puppeteer Library
 
puppeteer.launch({headless:true}).then(async browser => {
     const pageNew = await browser.newPage(); // Launch browser
     console.log('Step1 - Open Browser'); //Display message
     await pageNew .setViewport({ width: 1280, height: 800 })
     await pageNew .goto('https://techiescience.com/'); //Open LambdaGeeks
     //Capture Screenshot
     await pageNew .screenshot({ path: 'screenshot_lambda.png' });
     console.log('Step2 - Navigate LambdaGeeks and take screenshot');
     await browser.close();
     console.log('Step3 - Browser Closed');
 });

This code needs to be stored in the root directory of the Sample puppeteer project with the file name sample_script.js. Incase of Puppeteer-core, we need to include ‘puppeteer-core’ instead of ‘puppeteer’ at the very beginning of the script. For headful browser, we need to replace code “{headless:true}” with “{headless:false}”.

Step4# Execute Sample Puppeteer Script: The sample script can be executed from the command prompt using the below command –

npm node sample_script.js

After the execution, the screenshot will be capture and store in the root directory as “’screenshot_lambda.png”.

Puppeteer tutorial - Sample Puppeteer Project
Puppeteer tutorial – Sample Puppeteer Project

Now we will shown another sample Puppeteer script on amazon web application. This script will perform below steps along with verifications in each steps –

  • Invoke Amazon application.
  • Search a predefined book.
  • Add the searched book into cart.
  • Open cart and check if the book is available in cart.
  • Capture screen and close the browser.

We will only walk through the below script. We will learn in details about different steps to perform in next article. The sample script is shown below –

/**
 * @name Search in Amazon
*/
const puppeteer = require('puppeteer');
const reportPathDir = 'C:\\\\LambdaGeeks\\\\puppteer_proj_sample\\\\output\\\\';
const screenshotFile = 'screen1.png';
try {
  (async () => {
    
\t//Create browser and page object instance and navigate to the URL
    const browserWeb = await puppeteer.launch({ headless: false });
    const pageWeb = await browserWeb.newPage()
    await pageWeb.setViewport({ width: 1280, height: 800 });
    await pageWeb.goto('https://www.amazon.in/');
\t
\t//Enter the amazon Search criteria
\tlet searchBoxAmazon = await pageWeb.waitForXPath("//*/input[@id='twotabsearchtextbox']",{ visible: true });
\tif (searchBoxAmazon === null)
\t{
\t\tconsole.log('Amazon screen is not displayed');
\t}
\telse{\t\t
\t\tawait searchBoxAmazon.type("Testing Book");
\t\tconsole.log('Search criteria has been entered');
\t} \t\t
\t
\t//Clicked on search button
\tlet btnSearchAmazon = await pageWeb.waitForXPath("//*/input[@id='nav-search-submit-button']",{ visible: true });
\tif (btnSearchAmazon === null)
\t{
\t\tconsole.log('Search button is not showing');
\t}
\telse{
\t\tawait btnSearchAmazon.click();
\t\tconsole.log('Clicked on search button');
\t}\t
\t
\t//Click on specific search result
\tlet myBookAmazon = await pageWeb.waitForXPath("//*[contains(text(),'Selenium Testing Tools Cookbook Second Edition')]",{ visible: true })
\tif (myBookAmazon === null)
\t{
\t\tconsole.log('Book is not available');
\t}
\telse{
\t\tawait myBookAmazon.click();
\t\tconsole.log('Click on specific book to order');
\t} \t
\t
\t// Identify if the new tab has opened
\tconst pageTarget = pageWeb.target();
\tconst newTarget = await browserWeb.waitForTarget(target => target.opener() === pageTarget);
\t//get the new page object:
\tconst page2 = await newTarget.page();\t
\tawait page2.setViewport({ width: 1280, height: 800 });
\t
\t//Add to cart
\tlet addToCartAmazon = await page2.waitForXPath("//*/input[@id='add-to-cart-button']",{ visible: true });
\tif (addToCartAmazon === null)
\t{
\t\tconsole.log('Add to cart button is not available');
\t}
\telse{
\t\tconsole.log('Click on add to Cart button');
\t\tawait addToCartAmazon.click();\t\t
\t} \t\t
\t//Verify add to cart process\t
\tlet successMessageAmazon = await page2.waitForXPath("//*[contains(text(),'Added to Cart')]",{ visible: true });
\tif (successMessageAmazon === null)
\t{
\t\tconsole.log('Item is not added to cart');
\t}
\telse{
\t\tconsole.log('Item is added to cart successfully');\t\t
\t} \t
\t
\t// Capture no of cart
\tlet cartCountAmazon = await page2.waitForXPath("//*/span[@id='nav-cart-count']",{ visible: true});
\tlet valueCount = await page2.evaluate(el => el.textContent, cartCountAmazon)
\tconsole.log('Cart count: ' + valueCount);
\tcartCountAmazon.focus();
\tawait page2.screenshot({ path: screenshotFile });
\t
\tawait pageWeb.waitForTimeout(3000);    
\tawait page2.close();
\tawait pageWeb.close();
    await browserWeb.close();
  })()
} catch (e) {
  console.log(e)
}

Note: We will explain the details steps to write scripts in next articles.

Conclusion:

In this introductory article about “Install Puppeteer” from the “Puppeteer Tutorial”, we have explained about the detailed steps to install different Puppeteer packages from the scratch. The puppeteer setup includes different component installations such as, install NodeJs, install VSCode, install Puppeteer, create and execute Puppeteer sample project. In the next Puppeteer tutorial, we will explain detailed steps to use the Puppeteer as a web scraping tool. Please click  here to read from reference portal.

     

Leave a Comment