Tutorial: Build a React, Firebase and Google Cloud Vision machine learning API app
What technologies/libraries will I learn?
- Firebase (Firestore, Functions, Storage, Google Authentication, Hosting, database security, storage security)
- Google Cloud Vision machine learning API
- Google authentication with Firebase
- React (create-react-app)
- Serverless functions (using Firebase Functions)
- Async/await
What will I build?
Demo: http://doppelganger-app.firebaseapp.com/
Code: https://bitbucket.org/bochrane/doppel-app
Designs: https://drive.google.com/file/d/1H_c7IBZG5Ggvns1MmPPQn5Tqik5w3Cfb/view?usp=sharing
This video below shows what we will be building
You will make a react app using firebase functions that uploads a photo of yourself then returns a list of images of people that look just like you — according to who Google thinks you look like anyway :)
How long is this tutorial? 1–2 hours if you code along, about 20–30 minutes if you just want to read along
Having just built an enterprise app that books construction equipment for a building company using Firebase, I am so impressed with what it has to offer that I want to showcase some things Firebase has made possible that a few years ago would have taken months to build.
Note: If you wish you see the full working serverless functions talking to the Google Cloud Vision API you will need to enter later in the tutorial a valid credit card to Firebase — don’t worry though you won’t get charged until you use huge amounts of function calls which we won’t get anywhere near in this tutorial.
So how are we building this app and where do these doppelgänger images come from?
Here’s what we will use:
- Google Cloud Vision API (machine learning API for images) will provide a pre-trained model, analysing any image we upload and provide a list of images that look just like the one we upload — pretty neat!
- Firebase functions (yep we will be learning about serverless functions in this tutorial too — we’ll get to what they are shortly if you haven’t come across them yet)
- Firebase Storage (for storing our images we upload) and Firebase Firestore (the new realtime, no-sql database from Firebase)
- React for the front end app
- Google login (using Firebase’s built in authentication with Google)
I’ve also included the Sketch Design files as well as the Invision clickable prototype — I find it’s much easier working with these 2 tools to design and prototype out what I want to build ahead of time.
So there are a lot of things we get to cover in this tutorial and I’m excited to share them with you. After making this app, my biggest realisation is that the ability to create high scale apps using machine learning APIs just got a whole lot simple. Now to the tutorial…
We’ll split the tutorial up as follows:
- Set up the react app (we’ll use create-react-app)
- Configure our Firebase app
- Set up Google Authentication
- Setting up Firebase Storage, Firestore and Add An Image
- Setting up Firebase Functions to allow serverless functions to fire whenever we upload a new image
- Set up Google Vision API in Firebase functions to request similar images from it’s API
- Test our Google Vision API with our app
- Deploy to Firebase for a live working app — woo!
1. Set up the react app
Firstly, make sure you have Node installed (at least version 6)
We’ll be using Facebook’s create-react-app a nice starter kit for building react apps.
npm install -g yarn
npx create-react-app my-doppelganger-app
cd my-doppelganger-app
yarn startAfter running `npm start` you should see this screen
Next, make 3 folders inside of your src folder as follows:
mkdir src/features src/config src/helpersThen lets set up some dummy components for Login, Home and Album
cd featurestouch Login.js Home.js
Then in src/features/Login.js add a dummy component for now:
src/features/Login.js
import React, { Component } from 'react';export default class Login extends Component {
render() {
return (
<div>this is my Login component</div>
);
}
}
And do the same for the Home component:
src/features/Home.js
import React, { Component } from 'react';export default class Home extends Component {
render() {
return (
<div>this is my Home component</div>
);
}
}
Next, lets set up the routes for the app. First we will install react-router and react-router-dom:
yarn add react-router react-router-domIn src/index.js we will add them and set up our routes:
src/index.js
import React from 'react';
import ReactDOM from 'react-dom';
import './index.css';
import registerServiceWorker from './registerServiceWorker';import { Redirect, Route, Switch } from 'react-router';
import { Router } from 'react-router-dom';
import createBrowserHistory from 'history/createBrowserHistory';import Login from './features/Login';
import Home from './features/Home';const customHistory = createBrowserHistory();const Root = () => {
return (
<Router history={customHistory}>
<Switch>
<Route path="/login" component={Login} />
<Route path="/app/home" component={Home} />
<Redirect from="/" to="/login" />
</Switch>
</Router>
)
}ReactDOM.render(<Root />, document.getElementById('root'));
registerServiceWorker();
We should now see the Login component in our browser:
Next, we can test our routing works by manually going to the home route to make sure they are working:
Now that our routes are working, we can begin styling up our Login component:
We will use this png for the button.
We will upload this png file to Firebase Storage. So first, we will need to sign into firebase with a google account, then create a new Firebase app
Then go to Storage on the left navigation and press Get Started. You will then be shown the default security rules for your Storage bucket — below shows that the default security is to allow access to any document as long as the user is logged in. For our purposes we want to allow some images to allow logged out users to view our Google login button and background image the signin page.
Let’s change these rules to allow anyone to view any image for now (we will lock this down later).
In the Rules tab in Storage, change your rules as follows:
service firebase.storage {
match /b/{bucket}/o {
match /{allPaths=**} {
allow read, write;
}
}
}Next, upload your button image to your Storage
Next, copy the Download URL of the image you just uploaded by clicking the File location dropdown after clicking the image row
Now we can set our background image of the div we are about to make to the Google image like this:
src/features/Login.js
Above, we have included a loginWithGoogle helper method which uses the firebaseAuth().signInWithRedirect(googleProvider) method firebase gives us out of the box, and the googleProvider passed into this method is found in config.constants.js which is just a new instance of firebase.auth.GoogleAuthProvider().
We also show a simple loading screen while we are logging into Google using localStorage.
index.css
html {
margin: 0;
width: 100%;
}body {
margin: 0;
padding: 0;
font-family: sans-serif;
max-width: 768px;
margin: 0 auto;
height: 100vh;
width: 100%;
}.login-button {
height: 100px;
width: 100%;
position: fixed;
bottom: 0;
background: black;
max-width: 768px;
margin: 0 auto;
}.login-container {
background: #FFC107;
width: 100%;
height: 100vh;
}.button-text {
position: relative;
font-size: 25px;
color: white;
line-height: 50px;
left: 90px;
}.google-logo {
height: 50px;
float: left;
background-size: contain;
background-repeat: no-repeat;
margin: 25px 20px;
}#root {
width: 100%;
height: 100vh;
}
Next, we will create dummy data in the Home route to show what our doppelgänger feed will look like
First, we will make use of react-bootstrap for our grid system
yarn add react-bootstrapThen add the following to src/features/Home.js
src/features/Home.js
In Home.js above, you will see we have added React Bootstrap including Image, Grid, Row and Col components. We have also added a set of dummy images for now, and set it as our initial state of ‘allPhotos’.
Before our final return statement, we loop through the allPhotos we set in the initial state using the built it javascript “map” method. Each time we loop through a photo, we create a piece of html which dynamically includes the details about that image.
And add this css to index.css (this will be all the css we need for the app)
index.css
For the icons and react-bootstrap to work, we need to add the material icons cdn as well as the react-bootstrap css files.
index.html
<link rel="manifest" href="%PUBLIC_URL%/manifest.json">
<link rel="shortcut icon" href="%PUBLIC_URL%/favicon.ico">...<link href="https://fonts.googleapis.com/icon?family=Material+Icons" rel="stylesheet"><link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css" integrity="sha384-BVYiiSIFeK1dGmJRAkycuHAHRg32OmUcww7on3RYdg4Va+PmSTsz/K68vbdEjh4u" crossorigin="anonymous">...
We should now see our dummy list of photos as well as our bottom nav bar like this:
Our index.html should now look like this:
public/index.html
2. Configure our Firebase app
Next, let’s set up our Firebase app inside of our React app.
First, go to your firebase console and click “Project Overview” in the top left navigation.
Then click the “Add Firebase to your web app” button and copy the code that pops up in the modal popup.
Create a config file in src/config/constants.js
src/config/constants.js
import firebase from 'firebase';const config = {
apiKey: "AIzaSyBBaJIkRtF3jgzFF7BC83G6nI9joZKxezg",
authDomain: "doppelganger-app.firebaseapp.com",
databaseURL: "https://doppelganger-app.firebaseio.com",
projectId: "doppelganger-app",
storageBucket: "doppelganger-app.appspot.com",
messagingSenderId: "115334021919"
};firebase.initializeApp(config);export const googleProvider = new firebase.auth.GoogleAuthProvider();
export const firebaseAuth = firebase.auth;
export const db = firebase.firestore().settings({ timestampsInSnapshots: true });
Above, we have used the config details from our firebase app. We have also added the Google Auth provider from firebase that will allow us to log in via Google. And lastly we have set up our Firebase database — in this case we will be using Firestore which is the new realtime document store. There is a setting we append to the firestore setup which saves create_at timestamps as a firestore timestamp instead of a regular date object.
We will also need to install firebase as well as the firebase CLI (firebase-tools) as follows (I’ve chosen a specific version of Firebase tools to ensure the app works for you as it did for me but feel free to use the latest version if you’d like):
npm install -g firebase-tools@3.17.4We will also install firebase locally into our app:
yarn add firebase@5.0.2Next, let’s set up firebase locally in our terminal:
(note: I’m using Oh-My-Zsh for my terminal which makes the syntax a bit nicer and I’m using the Green theme for the standard Terminal app on Mac, just go to Terminal -> Preferences -> select Grass and click Default to use this theme)
firebase login
You will then get a Google login popup in a new browser — go ahead and authenticate with the gmail account you used to sign up for Firebase.
Once you are logged in via the terminal, we can set up that this app is a Firebase app as follows:
firebase initYou will be prompted to answer a few questions, use these answers:
- You will see 5 options, to tick the ones we want press the spacebar and up and down to move to the options we want. Press space bar for Firestore, Functions, Hosting and Storage (we won’t be needing Database as that is the old realtime database and we will be using the Firestore database)
2. Given we are logged in, we can now find the project we created in Firebase in the list on the next question:
3. Press Enter to accept the default for the following steps (these images below look similar but are different setup questions for our Firebase app:
And finally we are set up as a Firebase app — woohoo!
You’ll notice a bunch of new files and folders have been added. Very briefly and overview of these files is a follows:
a. .firebaserc file — this allows firebase to know which app to use and will give a name like default that will point to the firebase app you named in your firebase console
b. firebase.json — this lets firebase knwo where to find the rules files we can define in our app, and database indexes we create (firebase an pre-index our datebase for to make filtering quicker — we will go through this later in the tutorial) and also where the build folder exists that firebase will deploy to the web — with create-react-app we will run a command later of npm run build which will generate a build folder using webpack.
c. firestore.indexes.json — as mentioned this is where we can define our database indexes to make filtering on various collections pre-indexed and much quicker
d. storage.rules — we have changed our storage rules in the firebase console online, but once we deploy this app they will get overridden by the storage rules in this file, so for now lets change the file to reflect what we did in the firebase console. Just make sure your storage.rules file looks like this:
storage.rules
service firebase.storage {
match /b/{bucket}/o {
match /{allPaths=**} {
allow read, write;
}
}
}e. You will also see a new folder called functions. This is where our serverless functions will go — exciting times! We will get to these later in to the tutorial also.
So that’s the new files and folders, now we’ll go set up Firebase Storage to allow us to save some images.
IMPORTANT: Before we continue, if you are using git source code control like Github or Bitbucket to save your work, please ensure you add /functions/node_modules to your .gitignore file like below:
.gitignore
...# dependencies
/node_modules
/functions/node_modules# testing
/coverage...
3. Set up Google Authentication
Next we will set up being able to log in and log out of our app with Google using Firebase’s built in Google auth provider.
First, we will set up our auth.js helper file:
touch src/helpers/auth.jssrc/helpers/auth.js
import { firebaseAuth, googleProvider } from '../config/constants';export function loginWithGoogle() {
return firebaseAuth().signInWithRedirect(googleProvider);
}export function logout() {
return firebaseAuth().signOut();
}
The auth.js file above will create 2 methods for logging in to Firebase’s Google auth provider and also logging out too.
Next, we will update our Login.js file as follows:
src/features/Login.js
import React, { Component } from 'react';
import { loginWithGoogle } from '../helpers/auth';
import { firebaseAuth } from '../config/constants';const firebaseAuthKey = 'firebaseAuthInProgress';
const appTokenKey = 'appToken';export default class Login extends Component {constructor(props) {
super(props);
this.state = { splashScreen: false };
this.handleGoogleLogin = this.handleGoogleLogin.bind(this);
}handleGoogleLogin() {
loginWithGoogle()
.catch(err => {
localStorage.removeItem(firebaseAuthKey)
});// this will set the splashscreen until its overridden by the real firebaseAuthKey
localStorage.setItem(firebaseAuthKey, '1');
}componentWillMount() {// checks if we are logged in, if we are go to the home route
if (localStorage.getItem(appTokenKey)) {
this.props.history.push('/app/home');
return;
}firebaseAuth().onAuthStateChanged(user => {
if (user) {
localStorage.removeItem(firebaseAuthKey);
localStorage.setItem(appTokenKey, user.uid);
this.props.history.push('/app/home')
}
})
}render() {if (localStorage.getItem(firebaseAuthKey) === '1')
return <Splashscreen />;
return <LoginPage handleGoogleLogin={this.handleGoogleLogin} />;}
}// this is the URL we copied from firebase storage
const loginButtonUrl = 'https://firebasestorage.googleapis.com/v0/b/doppelganger-app.appspot.com/o/google-icon-white.png?alt=media&token=ff891c5f-f2a4-441e-b457-d71b9b21762f';const styles = {
backgroundImage: `url(${loginButtonUrl})`
}const LoginPage = ({ handleGoogleLogin }) => (<div className="login-container">
<div onClick={handleGoogleLogin} className="login-button">
<div style={styles} className="google-logo">
<span className="button-text">Sign In With Google</span>
</div>
</div>
</div>)const Splashscreen = () => (<p>Please Wait Loading...</p>);
We’ve added a lot of code here so let’s step through it.
- we imported the loginWithGoogle helper method
- in our constructor, we are hiding the splashscreen initially, and we are also binding out handleGoogleLogin function so we can use it in the component without the need to bind it from within the component
- the handleGoogleLogin function calls out to the pre-built loginWithGoogle function we imported (we aren’t interested in the success result, as it will go through to the home page on success anyway, so we are just catching the error for now)
- we are using the componentDidMount React lifecycle method to listen for when our component mounted, as we need to know if we have a logged in user. We are also using Firebase’s built onAuthStateChanged method which checks when things like a page reload occurs, and returns the currently logged in user
- we are using localStorage to check if the login is loading, in which case show the Splashscreen component, otherwise show the Login component
- we are using a functional component for the LoginPage and we are also passing in an argument to this component which is the handleGoogleLogin method for use when we click the login button so it has access to that method
Before we test this out, we need to go to our Firebase console and switch on Google as an authentication option for the app.
If you have already tried the Google auth in the app and it gets stuck on the Splashscreen, go into the console in your browser and remove the localStorage item that was just set:
In your browser console run this if it’s stuck on the Splashscreen:
localStorage.removeItem("firebaseAuthInProgress")In Firebase console, go to Authentication and click Set Up Sign In Method
Firebase console > Authentication > Sign-In Method > Google
Then click Google and toggle enable to on, then press Save
You should now get a Google authentication screen on pressing your Google Login button — woo hoo!
Let’s now get Sign Out working too.
In src/features/Home.js add the following throughout the code:
src/features/Home.js
...
import { logout } from '../helpers/auth';
...// inside of the constructor
constructor() { ...
this.handleLogout = this.handleLogout.bind(this);
...}handleLogout() {
logout()
.then(() => {
localStorage.removeItem(appTokenKey);
this.props.history.push("/login");
console.log("user signed out from firebase");
}.bind(this));
}
// then in the logout icon (the final Col in the Grid of icons) add a logout method like this:...
<Col onClick={this.handleLogout} xs={4} className="col-bottom">
<i className="bottom-icon material-icons">assignment_return</i>
</Col>
...
So Home.js should look like this:
import React, { Component } from 'react';
import { Image, Grid, Row, Col } from 'react-bootstrap';
import { Link } from 'react-router-dom';
import { logout } from '../helpers/auth';const appTokenKey = "appToken";
export default class Home extends Component {constructor(props) {
super(props);const allPhotos = [
{
id: 'randomstringimadeup43454356546',
url: 'http://fillmurray.com/200/200'
},
{
id: 'randomstringimadeup43523526534565',
url: 'http://fillmurray.com/200/200'
},
{
id: 'randomstringimadeup433245234534',
url: 'http://fillmurray.com/200/200'
}
]this.state = {
allPhotos
};this.handleLogout = this.handleLogout.bind(this);}handleLogout() {
logout()
.then(() => {
localStorage.removeItem(appTokenKey);
this.props.history.push("/login");
console.log("user signed out from firebase");
});
}render() {// our doppelganger images
const allImages = this.state.allPhotos.map(photo => {return (
<div key={photo.id}>
<div style={{minHeight: '215px'}}>
<i className="bottom-icon material-icons main-close">close</i>
<Image style={{ width: '100%' }} src={photo.url} responsive />
</div>
</div>
);
})return (
<div>
{allImages}<Grid className="bottom-nav">
<Row className="show-grid">
<Col xs={4} className="col-bottom">
<Link to="/app/album"><i className="bottom-icon material-icons">collections</i></Link>
</Col>
<Col xs={4} className="col-bottom">
<i className="bottom-icon material-icons">camera_alt</i>
</Col>
<Col onClick={this.handleLogout} xs={4} className="col-bottom">
<i className="bottom-icon material-icons">assignment_return</i>
</Col>
</Row>
</Grid></div>
);
}
}
Now you should now be able to log in and log out of your app with Google auth — nice work — and you know what…
4. Setting up Firebase Storage, Firestore and Add An Image
In this section, we will middle button in our bottom nav bar, when pressed to prompt us to upload an image which will get saved to Firebase Storage. Let’s go!
We will now use a file uploader package called react-firebase-file-uploader, so let’s first install it:
yarn add react-firebase-file-uploaderOnce we have this installed, add in the FileUploader JSX into Home.js where your middle nav bar button sits as follows:
FileUploader JSX snippet
We will also create the “handleUploadSuccess” function (which we can see is a function that gets executed in the FileUploader snippet above when the file has successfully uploaded. It will return a bunch of details about the image that it just uploaded to Firebase Storage, and we will save this image url to a new collection we are about to create called ‘photos’ in our Firebase Firestore database (also called a document store).
The handleUploadSuccess function will look like this:
handleUploadSuccess.js
Our Home.js file should now look like this after the 2 changes above:
src/features/Home.js
Let’s explain the “handleUploadSuccess” method (or function — which ever way you would like to call it) as well the FileUploader JSX snippet too.
- HandleUploadSuccess
- The handleUploadSuccess method gets kicked off from the prop on the FileUploader jsx snippet called “onUploadSuccess”
- We are using the ES7 async-await syntax, which does is a nicer way of doing promises. Instead of “then-ing” and “catch-ing” everything, we can now add the word “async” to the front of our functions, and put the word “await” in front of anything we would have usually wanted to add “.then” to. This is great to cleaning up our code and not have promises all over the place
- With the async-await syntax, we can now check for errors using a try-catch block. The try block wraps all of our async-await code, and if there is an error on any of the await pieces of code, it will throw that into the catch block with the error that occurred, which in this example I’ve called “err”
- I’ve also using destructuring syntax from ES6 which allows me to pull out specific parts of an object which you can see I’ve used to get the bucket and fullPath values in the snippet above. (the bucket and fullPath are metadata on the photo that describe where the photo is in our storage bucket — this is important to know where to access our photo from to display it in our app)
- With the photo info and the user info, we then create a “newPhoto” object to save into our Firestore collection which is called “photos”
Action required:
In order to get our app working and saving this new photo to not only the Firestore Storage, but also add this as an entry into our photos collection in our database, we need to set up Firestore from the Firebase console like this:
Go to Firebase console > Database
You will be asked what database security settings you would like — choose Start In Test Mode and press Enable. This will allow reads and writes from non-authenticated users — but given this is a test app we will leave it like this for now.
Now that we have enabled the database, let’s test out uploading a photo to the app by pressing the middle button in the bottom nav.
If successful, you should see some console logs as well as being able to look into both the storage and database in Firebase and see something like the following:
Browser console logs:
Storage:
Firestore photos collection:
Nice work!
The last part of this section is to get the list of photos we have saved in our database and show them on the screen. We are currently using a mock set of images (thanks Phil Murray — but your time in our app is up).
Let’s now used the componentDidMount React lifecycle method to ask for the images from our Firebase Firestore photos collection.
To get an initial set of photos, we will use the .get method (later for realtime updates, we will also include the onSnapshot method from Firestore too)
componentDidMount method:
Now our app should be able to show the images we upload like this (after a reload of course which is what we will fix next):
You’ll notice in out componentDidMount method we reach out to another method called getInitial which will get all of the photos for us. The nice part about getInitial is it’s a listener rather than a one time get of the photos. So any time we make changes to the photos collection it will automatically update our front end list of photos too. We are using Firestore’s built in onSnapshot method to allow realtime updates — woo!
You’ll also notice in the getInitial method we are ensuring we have a logged in user before we ask for photos. We do this with the built in Firebase auth method onAuthStateChanged to make sure someone is signed in.
Try adding another — and this time it should appear in the app once it hits the database — nice!
Great work! We are getting close to having a working doppelganger app. Let’s now get the serverless functions working, then hook them up to the Google Vision API — then we are done.
You Home.js file should now look like this:
src/features/Home.js
5. Setting up Firebase Functions to allow serverless functions to fire whenever we upload a new image
We are now going to use serverless functions to allow us to write server side code which will connect into the Google Vision API to check who looks like us.
If you haven’t looked into the serverless movement yet, there are some great starters over at https://serverless.com/learn/ to help you get started.
In a nutshell, the serverless movement is more than just writing serverless functions, it’s about allowing developers to focus more on business logic and less on infrastructure and setup. Serverless is a bit of a misleading name, because there are still servers that power your app. It’s just that as a developer a lot of that setup and scaling is taken care of (in the early days anyway — serverless has a lot of power but also comes with it’s own hidden gotchas like monitoring and cold starts which are improving every day).
I like to think about serverless functions being like any back end function we might write. It’s just each of these functions has their own little server underneath them that is waiting to be run.
When you run these functions, they will spin up their own mini container of sorts, usually for between a few milliseconds and up to 6 seconds and sometime longer depending on your function. Once you are done with that function executing, the mini container is shut back down. You are charged for the amount of memory used and milliseconds of execution time. You might hear people talk about serverless functions as Lambdas too given AWS Lambda was one of the first offerings (although not the first) to make serverless functions mainstream to developers).
Firebase functions are another serverless function offering using the Google Cloud Platform.
Ok, enough about serverless functions Ben, let’s just write some…
Hang on, why are we writing a serverless function again?
Because we want to talk to the Google Vision API to give us a list of images that look like whatever image it is we upload to our photos collection.
So here we go…
Firebase functions
If you can’t already see a functions folder in your main app, then let’s get you set up with a functions folder as follows from the firebase functions docs https://firebase.google.com/docs/functions/get-started (if you followed along much earlier in this tutorial and asked for functions in the setup this won’t apply)
firebase init functionsNext, we should see a folder called functions and inside of it a file called index.js
We won’t see much in this index.js file yet, but this is where we can write all of our serverless functions that will get deployed to the Google Cloud platform.
Our app has a package.json. Our function folder also has it’s own package.json which is separate again and are node modules we will need for our server side code we are about to write. Let’s set up the node modules we will need for our serverless functions:
cd functionsNext, add the necessary node modules, including Google Cloud Vision API node module
yarn add firebase-functions firebase-admin promise @google-cloud/visionThen install all modules that were already there with the new ones:
yarnWe will add the promise library as all functions require them to be promisified (that’s a word I swear?).
Then in functions/index.js we will set up our serverless functions file:
functions/index.js
Above, we see we’ve imported the modules we just installed an also initiated our Firestore database which we will need to update the images we get back from the Google Vision API.
Next, let’s write a function called addSimilarImages which will add similar images from Google Vision API into a list of image urls that we will save against the photo we just saved in Firestore.
Firestore has a great feature which allows for listening to events to the database. Below, you will see this in use through the functions.firestore.document(‘photos/{document}’)
.onCreate method as follows:
functions/index.js (the full version with setup and the function in it)
You can see above the following:
- Firestore uses an onCreate listener which allows for a function to be invoked when an event like adding a new photo to the collection happens
- The syntax allows us to check for any document in the photos collection by adding .document(‘photos/{document}
- snap gives back a snapshot of the data that was added, in this case the photo document that was added to the database, and context let’s us know what type of method is was, e.g. create, read, update destroy amongst other metadata
- we create a full photo url that the Google Vision API understands by recreating a google storage style url called photoUrl
- we then return a promise after going out to the google vision instance (visionClient) and using the webDetection method (there are a bunch more methods on this API such as labelDetection, faceDetection to name a couple that you might like to use instead which you can find here https://cloud.google.com/vision/docs/all-samples
- In this tutorial we are using the webDetection method, which picks up images that have similar attributes to the one uploaded (which we called photoUrl)
- We put this photoUrl through the Vision API and it returns a list of image urls — we are interested in the visually similar images
- We create an array of these visually similar images and update the document in the photos collection with the similarImages images
NOTE: Before the function above will work for us, we now need to enable the Google Cloud API on our account
6. Set up Google Vision API in Firebase functions to request similar images from it’s API
Now we are getting to the really exciting part where we will be using a machine learning service to help us figure who else in the world looks similar to us, according to Google anyway.
Step 1: Go to the Google Cloud console here https://console.cloud.google.com/ and login (don’t worry — you already have an account — you just need to log in using the same gmail account as your Firebase account)
Step 2: Click select a project then click New project
Step 3: Name your project (I just chose the same name as my firebase project but it’s ok if it’s different)
Step 4: Select the project you just made (there’s a trick here)
Once you’ve created the project, we are placed back into the main console page again, but in the top left, the project isn’t selected that we just create — so we need to select it.
You would think this is easy, but clicking Select a project, if you already have previous project may not show your newly created project in the list — you will need to click the All Projects tab next to the Recent Projects tab to find it.
Step 5: Enable Google Vision API
Go to the APIs & Services Dashboard — from the console go APIs & Service > Dashboard like below
Then click Enable APIs and Services like below
Then type “vision” in the search in this page below
And click the Vision API
Lastly, click Enable like below
Wowsa — there was bit in that! We are now ready to get into actually using a pre-trained machine learning API — woo!
7. Test our Google Vision API with our app and add the lookalike images to our app
This is it — the moment we’ve been working toward for however long you’ve been reading this on the bus, on the train, at home on the couch or at work :)
To test if this worked, we will need upload an image, then look at the logs on the Firebase function to see if we got some success. You will notice I’ve intentionally left in a bunch of console logs in the function so that we can see what data we are getting back.
Let’s go!
First, we need to deploy our functions to firebase in order for it to recognise the functions we’ve just written. We won’t deploy the front-end just yet, only our server side code.
To deploy the Firebase functions:
From your home directory of the app in your terminal:
firebase deploy --only functionsThis will only deploy the functions folder.
And hopefully you see a confirmation like this:
If you get any errors, just make sure you are logged in to firebase by using firebase logout, then firebase login (then log in with the correct gmail account) and run firebase use default (which is found in your .firebaserc file — it’s most likely you haven’t changed this so you should see a default key in there). Make sure also that you’ve run yarn in your functions folder too as the module must be installed before deploying to Firebase. And if you get an error like the top part of the image above, just try it again. Magic.
Allowing Cloud Vision to work requires billing to be added
Now as I mentioned at the start of the tutorial, if you want to see your functions work, we will need to include a valid credit card into your Firebase console. Don’t worry however, the free tier has huge monthly usage before Google start charging you a cent. You can check out the Firebase pricing here and the Cloud Vision pricing here. In short we would need to have more than 125 thousand function calls per day, or be hosting more than 5GB of photos, or do more than 20 thousand write operations to the photos collection per day or analyse more than 1000 images per month on Google Cloud Vision to start getting charged.
In the bottom left of your Firebase console, click Upgrade then select the Blaze Pay as you go plan.
Lastly, let go to our functions page in our Firebase console and check everything went through ok…
You can see above, our function which we called addSimilarImages is now in the list of functions in our firebase console.
Great — next let’s test out our function by uploading an image and then we will check the logs on this serverless function to see what happened…
I just uploaded a photo. Now let’s click View Logs like below in the Firebase Functions console:
If all goes well with our photo upload, in your function logs, you should see something like this:
You can see above that the function took 5553ms to execute and fetched a list of similar images from the Google Cloud Vision API — if you look closely in the logs above Google is suggesting I look like someone called Edwin from the Stripe team!
The last part of the tutorial is to show these similar images in the front end of our app. Let’s see who we look like according to Google…
First, we are going to add some loading spinners to let the user know we are waiting for some doppelganger images to come back from the Google Vision API. We will use react-spinners:
From the root of your app, in your terminal, run:
yarn add react-spinnersWe will then import this at the top of Home.js as well as import the Modal from react-bootstrap. We will use the modal for when someone clicks on an image of the doppelganger which will open up a modal with the image in it.
...import { Image, Grid, Row, Col, Modal } from 'react-bootstrap';...import { ScaleLoader } from 'react-spinners';
We are going to make a heap of small changes which I will run through below. At the end of the changes, here is what your Home.js file should look like:
src/features/Home.js
Let’s run through the additions we made:
- We’ve added in a row of loaders to show when we have just uploaded an image and are waiting for the doppelganger images to return from Google Cloud Vision. We loop through an array of “similarImages” which is attached to the photo document in the firebase function in index.js we made earlier
- We’ve also now added a list of doppelgangers below each image that has similarImages on it, and we’ve used some css to create a horizontal scroller for these images
- We’ve also added a modal, so that when a doppelganger image is clicked we can see an enlarged view of the image
- We also have a remove handler in the top left of each image to allow us to delete each image
- We added a where statement to our list of images, so that we only see the images we uploaded. In firebase, we can do this on the front end using .where(‘userId’, ‘==’, user.uid) in our collection call. In the next section we will also add Firestore rules on the backend which will add security to replicate what we have on the front end.
- We also added a small helper function to check if we are on mobile as I haven’t got around to fix a rotation issue when uploading from a mobile device — the workaround is to turn your device to landscape which allows the image to show up correctly when viewing it back on a desktop
- We’ve also added an imageRef piece of state — this becomes important for setting security rules on the Firebase Storage later on — it will allow us to match the user uploading an image to a specific file location
We have now got a working app — locally anyway. Let’s get this thing deployed to the web and finish this tute!
8. Deploy to Firebase for a live working app — woo!
The final part — deployment!
Step 1: Add security rules before we deploy
In your firestore.rules file, we will add the following to ensure only you can read, write and delete your photos.
firestore.rules
storage.rules
Each time we upload an image, we save the path to include the user’s auth id (aka userId).
If we try to request to upload an image without being logged in or another logged in user attempts to upload photos that reference your userId, it will deny the upload like this:
The rules above are a nice way of achieving attribute based access control — in simple terms being able to define what types of actions (create, read etc) users are allowed to do, and in what collections (e.g. photos collection). The rules are also functional, that is you can create functions outside of the service cloud.firestore block and refer to them inside of it too.
For our purposes, we are saying above that only the photos collections is given any access, and for any photo in the photos collection, we will allow a read or delete if your user id from firebase authentication is the same as the userId attribute on the photo document that we saved.
Similarly for writing objects, you can only save documents to the database is your user id is the same as the userId attribute on the payload of the photos collection document. Pretty nice I thought.
Step 2: Create a build version of our react app
In your terminal, run:
yarn buildIf it built correctly you should see a message like this:
** Important **
In your firebase.json file in the root of your app, we will tell firebase which folder to find the react app in. You will notice a build folder has been created to house our react app, so we need to change “public” to “build” like this:
firebase.json
{
"firestore": {
"rules": "firestore.rules",
"indexes": "firestore.indexes.json"
},
"hosting": {
"public": "build",
"rewrites": [ {
"source": "**",
"destination": "/index.html"
}],
"ignore": [
"firebase.json",
"**/.*",
"**/node_modules/**"
]
},
"storage": {
"rules": "storage.rules"
}
}Step 2: Deploy the full app to Firebase
In your terminal, run:
firebase deployThis will deploy both your front end app as well as the firebase functions folder.
With any luck your app is deployed and you have a working doppelganger app! Woo!
Well done on making it this far! We have covered a lot.
To summarise, you’ve just learnt how to:
- create a react app and use react-router-dom
- security rules for Firebase Firestore database, and Firebase Storage
- deployment of a react and firebase app
- interacting with a machine learning API with Google Cloud Vision
- serverless functions using Firebase Functions to talk to the Google Cloud Vision API
- and most importantly……
- You’ve found out who Google thinks you look like!
Please feel to leave any comments, questions and any improvements you’d like to see in this tutorial.
For any further questions, feel free to reach out to me on LinkedIn
