Thursday 9 June 2022

Integrating Google Cloud reCAPTCHA Enterprise in PHP

Google Cloud reCAPTCHA Enterprise is the successor (?) to reCAPTCHA v3. 

The code snippets Google supplies for integrating reCATPCHA Enterprise in PHP are frustratingly uncohesive and neither a working demo or complete code sample exists (as far as I can find at the time of writing). 

Moreover, the code snippets that are provided were incomplete for my implementation and the number of moving parts (Cloud projects, keys and credentials) is nothing short of overwhelming. 

Given the above, I struggled for some time just get everything working--which is where I'm up to now. So a disclaimer for those of you readers who are proficient PHP developers and who possess a deeper understanding of the Google Cloud Platform, please use this as a starting point only. There may be a better (more refined) approach. 

In terms of the official documentation and the code snippets I refer to above and will discuss in this post, start here: https://cloud.google.com/recaptcha-enterprise/docs/choose-key-type

I should also note I'm working in a basic web hosting environment with PHP installed. It's not a Google environment--think more budget web host ;)

Setup

Before you can get started, you'll need to login to the Google Cloud Platform, create a new project if you don't have a suitable already, and create a reCAPTCHA Enterprise key. Save the key somewhere and take note of your Cloud project ID while you're there as we'll need that later. You'll find your project ID on the relevant dashboard card. 

Incidentally, you can follow the link to reCAPTCHA Enterprise in the documentation to get to where you need to be in the Google Cloud Platform (or just navigate to Menu > Security > reCAPTCHA Enterprise). 

Once that's done, you may want to hop into Menu > APIs and services. If the reCAPTCHA Enterprise API isn't already enabled, find it in the library and enable it. While you're here, you'll also want to create credentials. This is where things are still a bit blurry for me but I created both an API key and a service account. The API key isn't used in my final code and it's possibly not necessary. I'm just used to API keys, I guess! 

The service account basically receives a name and ID and is then assigned the reCAPTCHA Agent role (locate reCAPTCAHA Enterprise in the list of products and services and then select the reCAPTCHA Agent role). 

Once the service account exists, I chose to create a key ("Manage keys"). Google will warn you about the risks of downloading service account keys and suggests using Workload Identity Federation--I don't yet know what this is so do you own homework here. I created a new key (ADD KEY > Create new key) as a .json file--you'll be prompted to download/save this file and I'd suggest storing a copy in a safe place as I have yet to find a way to download this same file again in future. 

Code

With that setup done, we now move into the code. The code runs clients side and server side and is comprised of your HTML page (including a form), a couple of scripts in the HEAD tag and some attributes on the form button. And of course the PHP code. 

Be sure to add your reCAPTCHA Enterprise key to both the script tag in the head and the data-sitekey attribute on your button. It's also required in the call to create_assessment() in the PHP code. 

I'll point out that, in my case, I want to trigger a score-based reCAPTCHA assessment when a user submits a form--no sooner and not unecessarily. I'm therefore using the relevant Google examples here, combined into a single, cohesive file. 

The HTML and script source is taken from this page: https://cloud.google.com/recaptcha-enterprise/docs/instrument-web-pages 


Note the PHP code is slightly modified to include the putenv call, which references the JSON key file we created earlier. You'll also want to drop in your Google Cloud Platform project ID in the call to create_assessment(). 

<html>
<head>
	<script src="https://www.google.com/recaptcha/enterprise.js?render=your_recaptcha_key"></script>
	<script>
	   function onSubmit(token) {
		 document.getElementById("demo-form").submit();
	   }
	</script>
</head>
<body>
<form id="demo-form" method="post">
<input type="text" />
<button class="g-recaptcha"
data-sitekey="your_recaptcha_key"
data-callback='onSubmit'
data-action='submit'>Submit</button>
</form>
</body>
</html>

<?php

require 'google-re/vendor/autoload.php';

use Google\Cloud\RecaptchaEnterprise\V1\RecaptchaEnterpriseServiceClient;
use Google\Cloud\RecaptchaEnterprise\V1\Event;
use Google\Cloud\RecaptchaEnterprise\V1\Assessment;
use Google\Cloud\RecaptchaEnterprise\V1\TokenProperties\InvalidReason;

/**
* Create an assessment to analyze the risk of a UI action.
* @param string $siteKey The key ID for the reCAPTCHA key (See https://cloud.google.com/recaptcha-enterprise/docs/create-key)
* @param string $token The user's response token for which you want to receive a reCAPTCHA score. (See https://cloud.google.com/recaptcha-enterprise/docs/create-assessment#retrieve_token)
* @param string $project Your Google Cloud project ID
*/
function create_assessment(
  string $siteKey,
  string $token,
  string $project
): void {
	
  // *** I added this line ***
  putenv('GOOGLE_APPLICATION_CREDENTIALS=your-service-account-key-file.json');
    
  // TODO: To avoid memory issues, move this client generation outside
  // of this example, and cache it (recommended) or call client.close()
  // before exiting this method.
  $client = new RecaptchaEnterpriseServiceClient();

  $projectName = $client->projectName($project);

  $event = (new Event())
	  ->setSiteKey($siteKey)
	  ->setToken($token);

  $assessment = (new Assessment())
	  ->setEvent($event);

  try {
	  $response = $client->createAssessment(
		  $projectName,
		  $assessment
	  );

	  // You can use the score only if the assessment is valid,
	  // In case of failures like re-submitting the same token, getValid() will return false
	  if ($response->getTokenProperties()->getValid() == false) {
		  printf('The CreateAssessment() call failed because the token was invalid for the following reason: ');
		  printf(InvalidReason::name($response->getTokenProperties()->getInvalidReason()));
	  } else {
		  printf('The score for the protection action is:');
		  printf($response->getRiskAnalysis()->getScore());

		  // Optional: You can use the following methods to get more data about the token
		  // Action name provided at token generation.
		  // printf($response->getTokenProperties()->getAction() . PHP_EOL);
		  // The timestamp corresponding to the generation of the token.
		  // printf($response->getTokenProperties()->getCreateTime()->getSeconds() . PHP_EOL);
		  // The hostname of the page on which the token was generated.
		  // printf($response->getTokenProperties()->getHostname() . PHP_EOL);
	  }
  } catch (exception $e) {
	  printf('CreateAssessment() call failed with the following error: ');
	  printf($e);
  }
}

   create_assessment(
      'YOUR_RECAPTCHA_SITE_KEY',
      $_POST['g-recaptcha-response'], // Safety first! Do you trust this code?
      'YOUR_GOOGLE_CLOUD_PROJECT_ID'
    );
?>

Drop all of the above in a file named whatever.php and upload it to your web server. Before any of this will run on your server, you'll need to attend to some dependencies. 

Upload your .json key file alongside your .php file (or put it somewhere else on the server and amend the putenv path in the PHP code). This file may warrant additional protections. 

With that sorted, you need to fetch all of the Google Cloud files the PHP above depends on. You'll likely want to do this using Composer (which needs to be installed on your desktop) and you can then run this command in a command window: 

composer require google/cloud-recaptcha-enterprise

I'll note I'm only fetching the reCAPTCHA Enterprise bits here--not the entire Google Cloud file set. You do you. 

I saved all of the Google files in a directory named google-re, which you'll see referenced in the first line our PHP. Adjust as required. 

A final note for the FileZilla users out there: when uploading the Google dependencies in particular, you may encounter a horrible runtime exception ("Fail to push limit") if you don't configure the FileZilla transfer type as Binary (Transfer > Transfer type > Binary). Refer to the answer to this question for more information: https://groups.google.com/g/protobuf/c/8_S93nJWxUE?pli=1

Run It

And now you should be able to request your page, submit the button. All being well, you'll encounter no exceptions and receive a nice meessage like "The score for the protection action is:0.89999997615814". Use that and other, related assessment information to act accordingly.  

I hope that helps someone!


Thursday 11 October 2018

Kdenlive won’t start on Windows

Trying to run kdenlive.exe on Windows 8.1 was silently failing (no UI error messages, etc). Turns out my antivirus (Avast!) was flagging one of the kdenlive files as a false positive (kioslave.exe) and I needed to add an exception through Avast!’s Virus Chest interface.

This issue was in relation to Kdenlive 18.08.1.

Tuesday 15 May 2018

Compare the checksum (hash) of a download file

Ever needed to compute the checksum (i.e. the hash) of a file you’ve downloaded from the internet? If not, you probably should—when you can—to ensure the file you’ve downloaded wasn’t tampered with (e.g. to insert malware) or corrupted while it was being downloaded.

As two examples, there are the large files (OS, etc) we download from the trustworthy MSDN library, which may face interruptions during download, and then there are those really handy utilities like WinDirStat that may originate from unsavoury locations on the internet.

Microsoft provides a handy tool for the purpose of computing the cryptographic hash value of one or more files called the File Checksum Integrity Verifier utility. It’s simple to use and can be found here: https://support.microsoft.com/en-au/help/841290/availability-and-description-of-the-file-checksum-integrity-verifier-u

Once extracted, usage is as simple:

fciv myfile.exe

But it can also compute MD5 and/or SHA-1 hashes for a directory of files and store results in a database. Full details can be found at the download link, above.

Sunday 8 April 2018

How to Save a Private Video from Facebook

Saving a video from your news feed or a group can be a little awkward because you’re operating within an authenticated environment (i.e. you log in to Facebook through your browser) but it’s actually pretty easy. Here are the steps:

  1. Browse to the video you want to download in Facebook and start it playing.
  2. Right-click the playing video and select Show video URL. Copy the URL and paste it into your browser’s address bar.
  3. Still in the address bar, change “www” to “m” to bring up the mobile version of the page; hit enter to load.
  4. In Chrome, hit F12 and click on the Network tab (ensure the network log is recording). I also like to check the Disable cache checkbox for convenience but if you don’t, click CTRL+F5 to forcefully reload the page and its resources from the server.
  5. Play the video (click play) and locate the the last request that pops up (it should be a .mp4 request).
  6. Right-click the request entry and click Copy > Copy link address
  7. Paste that address into your address bar and hit enter. The video will load.
  8. Right-click > Save As to save the .mp4 file

Tuesday 13 March 2018

Saving videos from a web site with VLC

I recently wrote about how to download and save a Brightcove video using the m3u8 format. I’ve since found an even easier method for other sites using other video formats.

It’s as simple as this:

  1. Open VLC and select Open Network Stream from the Media menu.
  2. Paste in the URL of your youtube.com (or similar) URL
  3. Click Play
  4. From the Tools > Codec Information menu, copy the full Location link to your clipboard
  5. Paste the copied location into Chrome and right-click the video/Save As

Depending on video quality, some videos (especially longer ones) may be on the large for your mobile device without additional transcoding.

If you have any problems with VLC complaining about not being able to play the video, you probably need to update VLC or the .lua or .luac file in the VLC install directory. I was experiencing MRL / 403 Forbidden errors until I updated the youtube.lua file.

To update:

  1. Save the text contents of this file as youtube.luac to a temporary location (note the .luac extension, not .lua): https://github.com/videolan/vlc/blob/master/share/lua/playlist/youtube.lua
  2. Replace the original youtube.luac in your VLC install directory (e.g. \VideoLAN\VLC\lua\playlist\ from c:\Program Files or c:\Program Files (x86)
  3. Restart VLC
  4. Update luac files for other sites if necessary

Sunday 11 February 2018

How to Download and Save Videos from a Website

I recently watched a free video embedded in a website and wanted to save the video itself for future reference (imagine the nerve!). The video content was delivered via the Brightcove player and was not a Flash/.swf file which I could otherwise deal with using traditional video download tools or shonky websites. The video was in fact, I believe, an HLS m3u8 video… admittedly, I don’t know if those are the same thing, or for that matter, what they are, and I don’t particularly care.

Rather than download a suspicious installer and infect my computer with the latest crypto virus, TubeOffline provided a handy and relatively simple tutorial using the ever trusty (and trustworthy) VLC media player. I’m reproducing the steps here in simple terms for future reference.

  1. Firstly, find the URL to the .m3u8 file you want to save. In Chrome, hit F12 and go to the Network tab. Check the Disable Cache button. Make sure the Record Network Log button is red and type “m3u8” (without quotes) in the Filter box. Load (or reload the page containing the video you’re after) and have a look at the network log. I grabbed the URL to the master.m3u8 file (right-click, Copy > Copy link address).
  2. From the Media menu in VLC, select Open Network Stream. It should open to the Network tab
  3. Copy the URL of the .m3u8 file to be downloaded and paste it in to the Network URL box. Don’t click the Play button!
  4. Click on the arrow next to the Play button and select Stream
  5. Click Next on the Source screen
  6. At the Destination screen, ensure the destination is set to File and click the Add button. Browse to the location where you want to save your downloaded video and specify a file name. The Save as type box will be left as Containers.
  7. On the Transcoding Options screen, leave Profile set to the default (Video – H.264 + MP3 (MP4). You can disable transcoding if you like but file sizes will likely be larger.
  8. On the Option Setup screen, leave the Stream all elementary streams checkbox un-ticked. Click the Stream button.
  9. VLC will begin saving the video to the selected file. Don’t try and play the video and don’t touch anything until the blue progress bar in VLC has reached the end (the far right side). You can sometimes monitor the file itself by hovering over in the file system to keep an eye on its size.

In practice, I noticed the blue progress bar reached the end and the file size was showing as 132MB. I thought it was all finished at that stage but opening the .mp4 file to review in a new VLC instance opened a whole lot of nothing with a duration zero seconds. After another minute or so, the file size bumped up to 133MB and opening the file again revealed the downloaded video complete with audio. I’ve found closing the original VLC instance seems to finalise the download/flush the last few bits to file (note: don’t just close the video or open another file—completely close VLC so it can flush its buffers and clean up). Your mileage may vary.

Tuesday 5 September 2017

How to save a video from Facebook

If you want to download and save a video from Facebook, right-click on the video and click the option to ‘Show video URL’; copy the link to your clipboard--it will look something like this:

https://www.facebook.com/user/videos/1026865037417235/

On a PC, paste the link into your browser’s address bar and replace ‘www’ with ‘mobile’ and hit go or enter. The URL will look like this:

https://mobile.facebook.com/user/videos/1026865037417235/

The mobile page will load and you can then right-click on the video and click ‘Save video as…’

Thursday 31 August 2017

How to access KeePass passwords from Chrome

Installing KeePassHttp with a browser extension isn’t difficult. Here are the simple steps:

  1. If your portable KeePass 2.x directory doesn’t have a “Plugins” directory, create it.
  2. Download the KeePassHttp.plgx file from GitHub (you’ll find a link to this file under the heading Non-Windows / Manual Windows Installation)—there’s no need to build, install Chocolately, etc. Save the .plgx file to your KeePass Plugins directory and restart KeePass if running.
  3. Add to the chromelPass extension to Chrome and click the button to connect. In KeePass, you’ll be prompted to name the key request which comes from the Chrome extension—I called mine chromelPass.
  4. Generate unique passwords to replace the single password you use everywhere.

Sunday 11 June 2017

PDF Toolkit (PDFTK) Examples

I started using PDFtk Free recently, which includes the PDFtk command-line tool PDFtk Server. Although PDFtk Free includes a user interface, it’s pretty basic unless you upgrade to PDFtk Pro (which admittedly doesn’t cost much).

As I’m on a scanning bent, I’ve got lot of PDFs to sort through and, for me, the command line is far more efficient than fiddling with a UI. The command line interface is also extremely simple to use.

Here you’ll find my boilerplate commands for accomplishing various tasks. All commands are non-destructive and create a new document rather than overwriting one of the input files.

Concatenate/Combine

Join a second document to the end of another PDF.

pdftk 1.pdf 2.pdf cat output out.pdf

Insert page(s) from another Document

Insert one or more pages from one PDF into a master document.

pdftk A=MasterFile.pdf B=FileToInsert.pdf cat A1-12 B1-7 A13-end output NewFile.pdf

Replace Page(s)

Replace one or more pages in a master document with specific pages from another document.

pdftk A=MasterFile.pdf B=FileToInsert.pdf cat A1-12 B1 A14-end output NewFile.pdf

Collate Scanned Odd and Even Pages (Even Pages in Reverse Order)

Interleave the pages from one document into another, where the pages in the second are reversed (for example when scanning both sides of a two-side document on a scanner that can’t flip pages automatically to scan the back of each page).

pdftk A=odd.pdf B=even.pdf shuffle A Bend-1 output collated.pdf

Extract Page(s)

Extract one or more pages from one PDF into a new document.

pdftk A=in.pdf cat A1-10 A15 A17 output out.pdf

Rotate 180 Degrees

Rotate one or more pages in a PDF 180 degrees. Use ‘east’ instead of ‘south’ to rotate 90 degrees instead.

pdftk in.pdf cat 1-endsouth output out.pdf

Sunday 16 April 2017

How to specify credentials to restart or shutdown a remote host with ‘net use’

I recently wanted to shutdown a server on my local network (notably a workgroup) using the Windows shutdown command but encountered an access denied error (or no feedback whatsoever).

I needed to use the administrator credentials of the remote server but shutdown.exe doesn’t accept credentials.

Using ‘net use’ to connect to the IPC$ share on remote host, you can set then call shutdown in that security context:

net use \\hostname\IPC$ mypassword /User:administrator
shutdown /r /t 60 /m \\hostname /f

In the net use command above, replace hostname with the name of your remote host (or maybe an IP address), replace mypassword with the password for the account you’re using, and replace administrator with the name of the local account on the remote host you wish to use (which presumably has the necessary rights to restart or shutdown the server).

In the shutdown command above, replace hostname with the name of your remote host, as per the net use command.

Note: Read the shutdown /? usage. The /r flag will restart the host; change /r to /s if you want to shutdown instead. Other parameters can also be specified. I also typically set the timeout (/t) flag to 0 (zero).

I call both commands back-to-back in a Windows Command Prompt, without running the console as Admin.

See below for a few more tips, links, and PowerShell alternatives.

If you’re used to operating as a domain user you may be asking why any of this is even necessary. It’s necessary in a workgroup context because the remote host has no idea who I am if I just tell it to shutdown from some other computer on the local IP network. It may be possible to an account on one host the necessary privileges on another host but I don’t know for certain and, in my limited experience, this can get hard and weird very quickly!

I used to run a domain controller on my small local network (just for fun) and all of my servers were members of a domain, which made security easy. I got rid of the DC as DHCP and other services were migrated to my internet router and to streamline things. In all regards, having one less server to manage, power, and so on is great. I’ll run a virtualised AD when I need to, e.g. for development purposes).

In addition to a media centre, I also still run a physical file server, primarily for backups. My basic process is to power on the file server once a week to backup the media centre, my laptop, etc and then shut it down again (so the kids don’t fiddle and to keep the noise down).

The backup process is otherwise seamless—the backup services on the file server and clients just pick up where they left off and run to completion without any human interaction (I use CrashPlan and would highly recommend it). I only need to power on the file server and then shut it down.

To shut down, I’d previously connect to the file server via Remote Desktop and then shut down. But that’s too hard. I can tell from the backup client on my laptop when a backup is complete so I just want to quickly shutdown the file server and go to bed.

In a domain environment, I could make sure my domain account was a member of the Administrators group on the file server and then call shutdown.exe from a command prompt on my laptop; without the DC, I need to explicitly provide shutdown.exe with the necessary credentials before actually calling shutdown.exe.

A few other notes: the file server is running Windows Server 2012 and my laptop is running Windows 8.1.

In recent versions of Windows (i.e. 2012+), I’ve read that UAC may interfere with things. You may want to turn it off but can simply add or configure this registry key (I didn’t have to):

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System
Value: LocalAccountTokenFilterPolicy
Data: 1 (to disable) or 0 (to enables filtering)
Type: REG_DWORD (32-bit)

You may need to restart after making this change.

Thank to https://helgeklein.com/blog/2011/08/access-denied-trying-to-connect-to-administrative-shares-on-windows-7/ for this one. See also this Microsoft help article (ID 951016) “Description of User Account Control and remote restrictions in Windows Vista”

Others suggested File and Printer sharing must be allowed in the Windows Firewall.

I believe PowerShell offers similar restart (and stop) commands. I haven’t tried them and can’t say whether they work with the techniques described here but I believe both accept a –credential parameter.

See these references:

https://msdn.microsoft.com/en-us/powershell/reference/5.1/microsoft.powershell.management/restart-computer

https://msdn.microsoft.com/en-us/powershell/reference/5.1/microsoft.powershell.management/stop-computer