# Introduction ## Smart Mirror [![Discord Chat](https://discordapp.com/api/guilds/258802311298547713/widget.png)](https://discord.gg/EMb4ynW) A voice controlled life automation hub, most commonly powered by the Raspberry Pi. ## Introduction This is the official documentation for the [smart-mirror](https://github.com/evancohen/smart-mirror), a voice controlled interface that controls your smart devices and displays information from a growing number of services. The smart mirror is powered by: * The [Raspberry Pi 3 or 4](http://amzn.to/2iU0kRn) * A webcam ([PlayStation Eye](http://amzn.to/2w5XjCy)) * Observation mirror (aka mirror pane) * Computer monitor The smart-mirror was originally inspired by [HomeMirror](https://github.com/HannahMitt/HomeMirror) and Michael Teeuw's [Magic Mirror](http://michaelteeuw.nl/tagged/magicmirror). It was originally created in a weekend and is now maintained by a growing community of contributors and enthusiasts. **Video Demo:** [**See it in action**](https://youtu.be/PDIbhV8Nvq8) {% embed url="" %} \> Note: The current video demonstrations do not display the mirror in its current state. Such as keyword spotting and remote configuration. Starting from scratch? No problem. Head on over to the [Hardware](/hardware) section to get started. If you encounter problems along the way check out the [Troubleshooting](/troubleshooting) section or join us in the [discord chat](https://discord.gg/EMb4ynW). Please file any issues or bugs [on GitHub](https://github.com/evancohen/smart-mirror/issues/new). ### About this documentation This documentation is constantly evolving. It is updated as we find issues and as we add new features. Who is the "we"? We are a community of people contributing, supporting, and improving this project. We are working to make the documentation as helpful, clear, and accurate as possible. Issues and/or concerns with the documentation? Please file an issue [on GitHub](https://github.com/evancohen/smart-mirror/issues/new). Commenting in line can cause readability issues for others. It is also difficult for anyone other than Evan Cohen to address or remove after resolving the documentation. > **This documentation outlines a sequential installation process. For successful installation and configuration you must follow it step by step. If you skip a step that seems insignificant it can cause issues down the line.** \##Language Translation If English isn't your first language, you can translate this site. function googleTranslateElementInit() { new google.translate.TranslateElement({pageLanguage: 'en', layout: google.translate.TranslateElement.InlineLayout.SIMPLE}, 'google\_translate\_element'); } #### Supported Platforms The smart-mirror is fully compatible with the following operating systems. Note that a small number of features require GPIO, devices without this will not be able to take advantage of these features. * ![](/files/-MHmHEsuMDXkNW3I1boy) Raspberry Pi OS * Pi 2 * Pi 3 * Pi 4 * ![](/files/-MHmHEsvHlPBMQGMmMW5) Linux (Most major distributions) * ![](/files/-MHmHEswlji_GDfoGYS_) OS X >= 10.8 #### Partially supported Platforms The smart-mirror is partially compatible with the following operating systems * ![](/files/-MHmHEsxGrixd_GZjM5m) Windows 7 / Server 2008 R2 or higher * Keyword Spotter is not supported. See [snowboy#31](https://github.com/Kitt-AI/snowboy/issues/31). * ![](/files/-MHmHEsyyd5Xt07zvh_r) iOS and Android (Experimental!) * See the `cordova` branch for details # Hardware ## Hardware To build a smart-mirror, you will need at least three things: * A two-way mirror * Glass: [Smart Mirror Kits (Amazon)](http://amzn.to/2wpOPsH) * Acrylic: [Supreme Tech (Amazon)](http://amzn.to/2wpP3jq) or [Tap Plastics](https://tapplastics.com/product/plastics/cut_to_size_plastic/two_way_mirrored_acrylic/558) * A monitor * Something to run the `smart-mirror` application. (Most people use a Raspberry Pi) In order to use the voice control features of your smart-mirror, you will also need a USB microphone (or USB Webcam w/ microphone). We highly recommend the [PlayStation Eye](http://amzn.to/2w5XjCy). You won't be able to use the webcam on it for anything because the driver is proprietary and it has an IR filter over the lens. However, for about $6 for a usb 4 microphone array with a name that rhymes with Raspberry Pi you can't beat it. Many have disassembled the PS Eye following directions on YouTube, so that it will fit better on their mirror. If you do this get 2, the first one is for practice. In addition, `smart-mirror` can control a Phillips Hue lighting system. Next Step: [Installation](/installation) See below for a quick guide to building a smart-mirror using mostly off-the-shelf parts: ## Building a smart-mirror Guide contributed by [Joel Hawksley](http://www.hawksley.org) ![The mirror installed](/files/pt2wbxmB6YxKDdumENUH) #### Parts list | Item | Price (incl. shipping) | | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------- | | [Two-Way Acrylic Mirror - 23-9/16" x 13"](http://www.tapplastics.com/product/plastics/cut_to_size_plastic/two_way_mirrored_acrylic/558) | $82.65 | | [HP 24uh 24" LED Monitor](http://amzn.to/2wkAH4N) | $129.99 | | [Glacier Bay Medicine Cabinet](http://www.homedepot.com/p/Glacier-Bay-15-1-4-in-x-26-in-Surface-Mount-Framed-Mirrored-Swing-Door-Medicine-Cabinet-in-White-S1627-12-B/100576352) | $34.97 | | [Keeper 24" Rubber Strap, 2 Pack](http://amzn.to/2xH0bJL) | $3.96 | | 4 x Eye hooks | $0.25 | | Misc. Cut Lumber | $10.00 | | Black Electrical Tape | $0.99 | | (Optional) Door Latch | $3.99 | | **Total** | $266.80 | #### Preparing the monitor I specifically chose the HP 24uh monitor due to its low cost, ability to fit in the pre-built medicine cabinet, and its downward-facing ports. It also turned out to be pretty easy to remove the monitor's bezel, without even having to open the casing. First, remove the single screw on the back of the monitor. Then, pop off the bezel using a flat-head screwdriver or similar implement. ![Removing the bezel](/files/VgiszBdaBJqbtrOuNcPl) Next, cover the exposed display frame with black electrical tape. The monitor will now be able to sit flush with the mirror. ![](/files/CtqH94F3FzAojbU650dn) The monitor, ready for installation in the cabinet: ![](/files/kPAznqK1I5kkwBRjJDat) #### Preparing the cabinet First, disassemble the door of the medicine cabinet, removing the mirror from the frame. The staples holding the frame together can be pulled out with needle-nose pliers. Replace the mirror with the two-way acrylic and re-assemble the frame. (I was able to re-use the staples, but added some wood glue to be safe) Next, tape over the bottom 1 3/4" of the back of the acrylic with electrical tape, masking off the area not covered by the monitor. ![Masking off the bottom of the mirror](/files/eVsaEfiEMQxsN1s7isAe) Add a strip of wood at the bottom of the frame to support the monitor. Attach an eye-hook to each end of the strip. Attach an eye hook in the top corners of the frame. ![](/files/QceEx9FhJM9fqUI5vYz9) Also, you might want to drill ventilation holes. I added three to the top of the cabinet: ![](/files/Ceme6xp8BkIoJbQz4J5Y) Due to the tight fit of the monitor, I was not able to use the magnetic closure that came with the cabinet. Instead, I installed a small latch: ![](/files/4Xx5NedTjbInICg3bbm0) ## Final assembly Place the monitor on the frame, connect cables/Raspberry Pi, and attach the rubber straps. ![](/files/1izrlGvQPprWhtyaXXbW) The final assembly, in profile: ![](/files/Ht5iaourGFVBEBUHjVMd) #### Mounting (Optional) Due to the somewhat heavy weight of the final assembly, I had wood cleats cut at my local hardware store, allowing me to anchor the mirror securely to the wall. ![](/files/rTFRQBM9nKhc98ex7vUK) That's it! Depending on your install location, you'll need to run some sort of power to the mirror. Enjoy your smart-mirror! ![](/files/pt2wbxmB6YxKDdumENUH) Next Step: [Installation](/installation) # Installation The easiest way to install the smart mirror is via the 1-line installation script for the Pi (or other systems) This script will install the smart mirror and it's dependencies: ``` curl -sL https://raw.githubusercontent.com/evancohen/smart-mirror/master/scripts/pi-install.sh | bash ``` Once that script finishes running (\~15 minutes on a standard connection) you'll want to [configure the mirror](/configuration). > Using another platform?\ > Something went wrong with the install?\ > Want to do things the old fashioned way? That's OK. That's why there are manual installation instructions: * [Install Raspberry Pi OS](/installation/installing_raspbian) * [Install dependencies and run](/installation/install_dependencies) # Install Raspberry PI OS **These instructions are specific to the Raspberry Pi 2 and 3** To get started I suggest a clean install of Raspbian. You can snag a fresh copy of Jessie w/ Pixel [Raspbian Download Page](https://www.raspberrypi.org/downloads/raspbian/). For instructions on how to install Raspbian see \[How To Install Raspbian(full)]\(/docs/howto/how\_to\_install\_raspbianfull.md). You'll also need to install Node (v6.x) which now comes bundled with npm. ``` curl -sL https://deb.nodesource.com/setup_8.x | sudo -E bash - sudo apt-get install -y nodejs ``` **Getting the code** Next up you'll want to clone this repository into your user's home folder on your Pi: ``` cd ~ git clone https://github.com/evancohen/smart-mirror.git ``` Next Step: [Install Dependencies](/installation/install_dependencies) # Install dependencies You'll need to install the following in order to run the keyword spotter and have the mirror listen to you: ``` sudo apt-get install sox libatlas-base-dev ``` Before we can run the thing we've got to install the projects dependencies. From the root of the `smart-mirror` directory run: ``` cd ~/smart-mirror npm install ``` This will take a few minutes, it has to download [electron-prebuilt](https://github.com/mafintosh/electron-prebuilt). Once it is complete please continue to configuring the smart-mirror. Next Step: [Configure the Pi](/configuration/configure_the_pi) # Configuration Before running the mirror you will have to configure both the pi and the mirror itself as well as obtain and configure your Chromium Speech API Keys: * [Configure the Pi](/configuration/configure_the_pi) * [Configuring Sound](/configuration/configuring-sound) * [Configuring Voice](/configuration/configuring_voice) * [First Time Running Smart-Mirror](/configuration/first_time_running_smart_mirror) * [Configure the smart-mirror](/configuration/configure_the_mirror) # Configure the Pi #### Rotate your monitor To rotate your monitor you'll need to add the following line to **/boot/config.txt** using the following command `sudo nano /boot/config.txt` ``` display_rotate=1 ``` You can also set this value to '3' to have a flipped vertical orientation. #### Disable screensaver and remove panel Edit the **/home/pi/.config/lxsession/LXDE-pi/autostart** file with `nano /home/pi/.config/lxsession/LXDE-pi/autostart`. **Reccomended** To disable the screensaver you'll want to comment out (with a '#') the `@xscreensaver`. You'll also want to add the following lines to that same file ``` @xset s off @xset -dpms @xset s noblank ``` **Optional** To remove the panel at the top of the screen to comment out the `@lxpanel` lines. If you want to be able to easily access the "menu" at the top of the screen do not do this step. #### Hide the mouse when inactive ``` sudo apt-get install unclutter ``` Then Add `unclutter -idle 0.1 -root` to **/etc/xdg/lxsession/LXDE-pi/autostart** with `sudo nano /etc/xdg/lxsession/LXDE-pi/autostart` Next Step: [Configuring Sound](/configuration/configuring-sound) # Configuring Sound > If you have trouble setting up speech recognition try heading over to the [troubleshooting section](/troubleshooting). If that doesn't work drop into the [discord chat](https://discord.gg/JDnHaZH). Hooray! Configuring sound on the mirror is now way easier. When you run the mirror for the first time, you'll be prompted to configure your input device base on available USB devices. Go ahead and skip to [Configuring Speech Recognition](/configuration/configuring_voice). ## Audio Output Should you want to change the output from AUX (headphone jack) to HDMI or back again you can run the following: To output audio through the headphone jack: ```bash amixer cset numid=3 1 ``` To force the audio back through HDMI you can run: ```bash amixer cset numid=3 2 ``` Next Step: [Configure Speech Recognition](/configuration/configuring_voice) # Cloud Speech Recognition > If you have trouble setting up speech recognition try heading over to the [troubleshooting section](/troubleshooting). If that doesn't work drop into the [discord chat](https://discord.gg/JDnHaZH). The smart mirror uses \[Sonus]\() for speech recognition. The following steps will get you everything you need to make this work: ## Setting up Speech Recognition The smart mirror uses [Sonus](https://github.com/evancohen/sonus) with Google Cloud Speech for keyword spotting and recognition. To set that up, you'll need to create a new project in the Cloud Platform Console: 1. In the Cloud Platform Console, go to the Projects page and select or create a new project.\ [**GO TO THE PROJECTS PAGE**](https://console.cloud.google.com/project) 2. Enable billing for your project.\ [**ENABLE BILLING**](https://support.google.com/cloud/answer/6293499#enable-billing) 3. Enable the Cloud Speech API.\ [**ENABLE THE API**](https://console.cloud.google.com/flows/enableapi?apiid=speech.googleapis.com) - For more info see [Cloud Speech API Pricing](https://cloud.google.com/speech/#cloud-speech-api-pricing) (for normal smart mirror usage it will be free) 4. Create a new **JSON service account key** and download it to the `smart-mirror` folder...\ [Credentials Guide: On Your Own Server](https://googlecloudplatform.github.io/google-cloud-node/#/docs/google-cloud/0.42.2/guides/authentication#onyourownserver). When prompted to create a new service account select "Project Owner" > New service account Keep this information for later. You'll need your projectID and keyfile to [configure the smart mirror](/configuration/configure_the_mirror#speech). ## Setting up your API key After setting up your **project** you will also need an API key for * geolocation * geocoding * Javascript Static Map * youtube (if you intend to use this function on your mirror) google apis. #### from the 3 bar menu on the left, select APIs and Services, then Library enter part of the name of each api in turn, and select it, and click enable repeat for the other apis listed after completing the enabling for all the listed apis * from the 3 bar menu on the left, select APIs and Services, then Credentials * in the **Create credentials** dropdown, select "API key", * save the value that is shown on the message window that pops up The API key should be a long string, something like this 'AIzaSyBGWRB2oC1P\_xYteMMsdTRV3Kv3aEBPQg' ## \[Optional] Train your own Keyword **Note:** This is no longer required, the mirror comes with a pre-trained universal model! If you're having trouble with the universal model you can follow these steps Training your own keyword will drastically increase the accuracy of keyword detection. This will be most accurate if you do this on your Pi using the microphone that you'll be using to trigger the mirror. **From the smart mirror directory run:** ### as of Mid 2019 local model training no longer works see **you can train your model here:** \[) Once trained, download the model and save it to the root of the smart-mirror directory. Next Step: [Run The Mirror For the First Time](/configuration/first_time_running_smart_mirror) # First Time Running Smart Mirror The smart-mirror is configured using the Remote Configuration Tool. This requires you starting the mirror. Open the terminal and type: ``` npm start or pm2 start smart-mirror (if you selected yes to use pm2 to manage starting during install) ``` If you're running the mirror for the first time (or for the first time since running upgrading to this version of the mirror) you'll see a QR Code with a URL under it. From a phone or another computer (on the same network as your Smart-Mirror) you can open a browser and manually enter the URL. If you're not running the mirror for the first time and you've properly configured the Sound and Voice, say the keyword/hotword and then "Show Remote Link" to display the URL to reach the Remote Configuration Tool. After going to the Home page click on Settings > Configure the Mirror. Next Step: [Configure the smart-mirror](/configuration/configure_the_mirror) # Configure the smart-mirror The smart-mirror is configured using the Remote Configuration Tool. This requires you starting the mirror. Open the terminal and type: ``` npm start or pm2 start smart-mirror (if you selected yes to use pm2 to manage starting during install) ``` There are 2 ways to find the IP and port of the remote for the mirror: ### From the command line When the mirror starts it will output the remote IP and port to the command line: ``` > smart-mirror@0.0.27 start /Users/evan/Git/smart-mirror > electron main.js Remote listening on http://192.168.1.130:8080 ``` ### From a QR code If you're running the mirror for the first time (or for the first time since running upgrading to this version of the mirror) you'll see a QR Code with a URL under it. From a phone or another computer (on the same network as your Smart-Mirror) you can open a browser and manually enter the URL. If you're not running the mirror for the first time and you've properly configured the Sound and Voice, say the keyword/hotword and then "Show Remote Link" to display the URL to reach the Remote Configuration Tool. ## Update Settings After going to the Home page of the remote, click on Settings > Configure the Mirror. It is **required** that you fill out Speech Settings! See below for links to get service keys and example values for config properties. You will have to obtain keys for the following functions Weather, YouTube, SoundCloud, not giving out new apikeys Fitbit, Geolocation, Geocoding Maps, Spotify, Last.FM/Scrobbler If you're using Hue Lights with this project you'll need to know the IP address of your Hue Hub and a username. Please go through this slowly, and thoroughly. Any issues or questions please join us on [discord chat](https://discord.gg/JDnHaZH). #### Index * [Language](#language) * [Speech](#speech) * [Layout](#layout) * [Greeting](#greeting) * [weather](#weather) * [Geolocation](#geolocation) * [Hue](#hue) * [Calendar](#calendar) * [Giphy](#giphy) * [YouTube](#youtube) * [SoundCloud](#soundcloud) * [Stock](#stock) * [Traffic](#traffic) * [TV Service](#tvsservice) * [AutoTimer](#autotimer) * [Motion](#motion) * [Fitbit](https://github.com/evancohen/smart-mirror/blob/master/Fitbit-README.md) *(See issue* [*#350*](https://github.com/evancohen/smart-mirror/issues/350) *before attempting configuration for fitbit)* ### Language The following languages are fully supported: * `en-XX` - English * `de-XX` - German * `es-XX` - Spanish * `fr-XX` - French * `ko-XX` - Korean * `pt-XX` - Portuguese Specific locales can also be specified, by replacing the `XX` above with the country code. For instance `en-US` for English (United States), `es-AR` for Spanish (Argentina), or `es-BO` for Spanish (Bolivia). For more details about supported speech detection languages see this [Google Cloud Platform Speech API](https://cloud.google.com/speech/docs/languages) page. ### Speech The speech config object has the following properties: * `keyFilename` - The location and name of your JSON keyfile for * `hotword` - The text of the hotword that you are using to trigger the mirror. This should be "Smart Mirror". Additional Keywords can be entered by clicking the `+` sign. * `model` - The filename for your model (should not include spaces). Additional models can be entered by clicking the `+` sign. * `sensitivity` - Sensitivity for the keyword spotter. This value is between 0 and 1. If you are getting too many false positives or are having trouble detecting you can change this value. ### Layout You can set these values to be `"main"` (recommended) or `"icesnow"`. ### Greeting `greeting` can either be an array of greetings to randomly select from OR it can be an object that specifies multiple arrays to choose from based on the time of day. **Disable Greeting** You can disable the greeting by selecting `Randomly All Day` for "How would you like greetings displayed?" and then clicking `-` sign to remove any values listed. **Randomly Selected Greeting** Select `Randomly All Day` for "How would you like greetings displayed?" and enter as many greetings as you would like by pressing the `+` sign. You can also click on a tab to rearrange the order. Lastly you can click the `-` sign to remove the greeting on the bottom. **Randomly Selected Greeting Based On The Time Of Day** Select `By Time of Day` for "How would you like greetings displayed?" and enter as many greetings as you would like for each time of day (Morning, Midday, Evening, and Night) by pressing the `+` sign under each heading. You can also click on a tab to rearrange the order. Lastly you can click the `-` sign to remove the greeting on the bottom. ### weather You'll need a an API yet currently Darksky is no longer giving out free api keys. existing keys will expire sometime near the end of 2021 we have two additional services Climacell or OpenWeather all services require an api key * `API Key` - see the links in the `API Key source` dropdown * `Units` - It should be ok to leave the `units` set as auto because the units that are used are determined by your location. if u want to force a particular unit setting select one from the dropdown (US or si) * `Refresh Interval (minutes)` - This is how often you would like the Weather to update in minutes. ## note that some services limit the number of calls allowed per day , the dropdown provides some reasonable choices from 5 to 60 minutes ### Geolocation Starting in 2019, the Google Geolocation API used for this feature **requires** an API key. The entry of your Latitude/Longitude location is still **optional** even tho the API key is **required**. See the API Key steps in [Configuring Voice](/configuration/configuring_voice) This API key also is used in the Map feature Entering the latitude and longitude is for people who are having issues with the smart-mirror's built in geolocation. You can override your latitude and longitude by entering them here. ### Geocoding starting is 2021, the service we used for locating which country you were in for weather units has stopped running, google provides a similar service, but needs an api enabled , add this to the apikey used for geolocation above, this API is **required**. ### Hue You'll need two things to set up your Philips Hue configuration, an `ip` and a `username`. You can find the instructions for this on the Philips Hue Documentation site in the [Getting Started](http://www.developers.meethue.com/documentation/getting-started) section (unfortunately you need to create an account to view this info). Optionally you can create groups (using the API for Philips Hue app) that you can control from the mirror by name. By default group `0` will control all the lights. ### Calendar You can have the mirror display your iCal's from Google Calendar, Outlook, iCloud, and more by adding them to the `icals` array. Note that these URLs should begin with `http(s)://` and not `ical://` There are two other properties: * `maxResults`: Maximum number of upcoming calender events to display. * `maxDays`: Maximum number of days to look into the future when listing upcoming events. ### Giphy If you want to display gifs on your mirror you can do that too! In the [Giphy Beta API](https://github.com/Giphy/GiphyAPI) the key is fixed `dc6zaTOxFJmzC` ### YouTube You can find instructions for getting YouTube API Keys here: The key will look similiar to this: `vy2u1t34bo123bu41234yduv1234tb` ### Stock You'll need a AlphaVantage API Key, which you can obtain from: \[) * `API Key` - After selecting from the provided fields, and entering your email address, you can find your key at the bottom of the page. It should look something like this: `QKQYHF247BBS6Q3V`. Enter this in the `key` field under `Stock Settings, Alpha Vantage API Key` ### SoundCloud SoundCloud API keys can be obtained from your app profile (this requires an account): The key will look similiar to this: `vy2u1t34bo123bu41234yduv1234tb` ### Note: Soundcloud is currently not granting new api keys (Jan 2020) ### Traffic Using your key from the [Bing Maps Portal](https://www.bingmapsportal.com/Application) you can specify an array of `trips` and a `reload_interval` (how often should the mirror refresh trip data, in minutes). A trip has the following properties: * `mode` - Mode of transportation. One of * `"Driving"` - By Car * `"Transit"` - By public transportation * `"Walking"` - Walking * `origin` - The address for the start of your trip * `destination` - The address for the destination of your trip * `name` - Human readable name for the destination * `startTime` - Time to start displaying on Smart Mirror. (optional: leave blank to always display) * `endTime` - Time to end displaying on Smart Mirror. (optional: leave blank to always display) If any of your trips aren't showing up it's likely because Bing Maps can't find the address you specified. Using a full postal address should fix this issue. Now that you've configured everything you're ready to save your configuration by clicking save. This will restart the Smart-Mirror. Next Step (OPTIONAL): [Setting up Smart-Mirror to Run on Boot](https://github.com/evancohen/docs.smart-mirror.io/blob/master/docs/setting_up_smart-mirror_to_run_on_boot.md) ### TV Shows Simply list the TV shows you would like to display on the mirror and the Mirror will display when the next episode airs if the information is known. ### Last.FM (the plugin is called scrobbler from the name Last.FM defines as the collector) this plugin requires an apikey from your Last.FM logon, see ### AutoTimer Setting up the mirror to go to sleep is easy. Just enter the mode, wait time (in minutes), if you want the mirror to turn on at the same time everyday enter the autoWake time in 24 hr format. **TV Mode** TV mode will just make the screen go dark... it doesn't actually "power off" this is used on TVs hence being TV mode. Many TVs will show a "no input" message of some sort when using monitor mode. So this makes it work for the people using that type of display device. **Monitor Mode** Monitor Mode sends a "sleep status" to the screen and stops sending a signal. In most monitors that will have the monitor go into a "sleep mode". If you're getting a "no input" message of some sort. Change the Mode to TV and see how that works. > When in monitor mode you must also fill out the commands to go to sleep or to wake the mirror. the default are as follows: > > For "Command used to wake up Smart Mirror" > > ``` > sudo ./scripts/raspi-monitor.sh on > /dev/null 2>&1 > ``` > > For "Command used put Smart Mirror to sleep" > > ``` > sudo ./scripts/raspi-monitor.sh off > /dev/null 2>&1 > ``` ### Motion Motion allows you to enable a PIR device on your Raspberry Pi. Please refer to the detailed instructions for [Enabling Motion Detection](/how_tos/enabling-motion-detection). you can also allow an external service of some sort to signal motion, if you have somethign else. an Example is the use the linux [Motion project](https://motion-project.github.io/motion_guide.html) to manage alerting to motion from a camera, by selecting External as a choice. the external service needs to call the provided /home/pi/smart-mirror/scripts/external\_motion script with a parameter started - signal motion detected ended - motion no longer detected * ### note that the Motion project signaller runs as root, so the full path to the script above will need to be specified ### Plugin Location information starting in version 0.27, a new configuration feature allows one to position plugin output in any of the defined locations via dropdown selection, and also disable plugin display if not wanted if new plugins are installed (there are some), the default display location is bottom center, just above the bottom bar info. to place the modules, create a new entry (+) fill in the plugin name (the folder it is in) and select the location where it should be displayed... then press submit to restart smart-mirror with this layout.. you can change the layout as many times as you wish, by changing the location dropdown for any/some/all plugins and then pressing submit.. ## Note: one this to note: if you uncheck the Active checkbox on a currently running plugin, its configuration information will be lost.. and will have to be entered again, it the Active checked is selected in the future.. ### save any info. # Running * [Setting up Smart-Mirror to Run on Boot](/running/setting_up_smart-mirror_to_run_on_boot) * [Commands Used to Run Smart-Mirror](/running/commands_used_to_run_smart-mirror) # Setting up Smart-Mirror to Run on Boot Optionally, you can configure your Pi to start the mirror on boot. During install you will be prompted if you want to use the node process manager [pm2](https://pm2.keymetrics.io/docs/usage/quick-start/) to control auto starting your smart-mirror. this process is setup for all platforms and hides any platform specific details. after starting your pi, you can use `pm2 help` for additional commands, stop, restart, logs, Have an issue? Take a look at the [Troubleshooting Page](https://github.com/evancohen/docs.smart-mirror.io/blob/master/troubleshooting.md). # Commands Used to Run Smart-Mirror Please do not use this page as part of the step by step process. This is more of a quick reference for use on gitter chat. Often people ask how to do these tasks and sending this page can help. #### Starting Smart-Mirror normally ``` npm start ``` #### Starting Smart-Mirror with the dev console ``` npm start dev ``` #### Training a Keyword Spotter Model ``` npm run train-model ``` #### Debugging microphone issues ``` npm run microphone-debug ``` # Troubleshooting & FAQs Many issues are resolved by going back and closely following the steps in the documentation. Often a step is missed, or admittedly a piece is unclear in the documentation. If you find something is unclear. Please let us know [on GitHub](https://github.com/evancohen/smart-mirror/issues/new). We are also available on [discord chat](https://discord.gg/JDnHaZH). Remember that you can search the documentation at the top of the sidebar on the left (or in the hamburger menu if you are on your phone). * [Issues installing electron-prebuilt](/troubleshooting/npm_install_issues) * [Microphone and Speech Recognition issues](/troubleshooting/microphone_and_speech_recognition_issues) # Issues installing electron-prebuilt You must run `npm install` when logged into the GUI and not over ssh as electron-prebuilt will not install outside of the GUI. Have a different issue? Take a look at the [Troubleshooting Page](/troubleshooting). # Microphone and Speech Recognition issues Most of these issues can be fixed by following the following: 1. Have you [installed all necessary dependencies](/installation/install_dependencies)? 2. Is your microphone [configured correctly](https://github.com/evancohen/docs.smart-mirror.io/blob/master/docs/configure_the_pi.html#audio-input-and-output)? 3. Have you enabled billing in [Google Speech Platform](/configuration/configuring_voice)? If you've done these and are still having issues I would recommend running `npm start dev` and seeing what (if any) error you get after you say "smart mirror". ### If saying "smart mirror" does nothing or you see a light glowing on the bottom but no other command works: There is likely an issue with Sonus. You can run it from the `smart-mirror` directory with: ```bash npm run sonus ``` **Testing Speech Using Sonus** When you start Sonus in the terminal, you will see this: ``` $ npm run sonus > smart-mirror@x.x.x sonus /home/bmartin/smart-mirror > node sonus.js █ ``` When you see the blinking cursor, this means that Sonus is listening. Now you would say the command, "smart mirror, show me how to tie a bow tie" You're results should be similar to this: ``` !h: 1 !p: show !p: show me !p: show me a !p: show me how !p: show me how to !p: show !p: show me !p: show me how to !p: show me how to !f: show me how to tie a bow tie ``` #### Let's break these results down **Confirming the Hotword is Detected.** When you say "smart mirror" if you've installed all dependencies correctly, trained your model, and configured the Speech Settings correctly on the Remote ConfigUI. You should get a `!h: 1` response. This is saying that the hotword was detected correctly. If you have more than one hotword you'll get a response `!h: x` where `x` is the index number of the hotword specified in the Remote ConfigUI. If you don't see anything displayed when saying "smart mirror" then the most common issues are: * `~/.asoundrc` isn't configured properly. [Follow steps to configure sound here.](/configuration/configuring-sound) * Personal Model file (commonly named `smart_mirror.pmdl`) is missing from the `smart-mirror` folder, or isn't entered correctly on Remote ConfigUI. This file can be named anything you would like as long as there's no spaces, and it is correctly entered on Remote ConfigUI. [Follow Steps for Speech Settings here.](/configuration/configure_the_mirror#speech) * Personal Model isn't trained. [Follow steps for training your own model here.](/configuration/configuring_voice) * Dependencies aren't properly installed. [follow steps here.](/installation/install_dependencies) **Confirming Google Speech API is configured correctly.** Assuming you're getting a response that the hotword is detected. You then would say the command. In our example above that command is "show me how to tie a bow tie". While the audio is streaming to Google Speech API you will get partial results. This is shown in the examples starting with `!p:` followed by the partial transcribed result. After it has completed transcribing the command a final results response is received. This is shown in the examples starting with `!f:` followed by the final transcribed result. If you're getting partial and final results, then speech recognition is working for you. If you see the response for the hotword, but no partial or final results, then the most common issues are: * Google Cloud Speech API key JSON file (commonly named `keyfile.json` is missing from the `smart-mirror` folder, or isn't entered correctly on Remote ConfigUI. The `keyfile.json` file can be named anything you would like as long as it has no spaces, and it is correctly entered on Remote ConfigUI. [Follow Steps for Speech Settings here.](/configuration/configure_the_mirror#speech) * Billing is either not enabled, project is not attached to the billing account, or billing account is closed within the Google Cloud Platform. [Follow the steps for configuring voice here.](/configuration/configuring_voice) **Still having issues?** Check out [recent audio related issues on GitHub](https://github.com/evancohen/smart-mirror/issues?utf8=%E2%9C%93\&q=is%3Aissue%20audio%20-label%3A%22status%3A%20Outdated%20Issue%20-%20Informational%20Only%22%20). **Also,** we're available on [discord chat](https://discord.gg/EMb4ynW) to help assist you in real time. # Issues with Remote and ConfigUI ### Only "Loading..." displayed on ConfigUI Page Some people have stated that the ConfigUI page just says loading. If you're having this issue you must use a newer browser. Most often this happens if you have an outdated version of Raspbian. If your version of Raspbian predates Raspbian Jessie W/PIXEL then we suggest following the steps in [How To Install Raspbian(full)](/how_tos/how_to_install_raspbianfull). Following this solution has fixed this specific issue 100% of the time. Have a different issue? Take a look at the [Troubleshooting Page](/troubleshooting). # Development and Contributing > ## `NOTE: This is a work in progress.` You can keep up with development on ## Contributing Everybody is invited and welcome to contribute to the smart mirror. There is a lot to do... If you are not a developer perhaps you would like to: * **Help with the documentation** on [docs.smart-mirror.io](http://docs.smart-mirror.io/), * **Localize the smart mirror** in a new language language (or improve an existing one) * **Helping others** on [Discord](https://discord.gg/EMb4ynW). If you are a developer and have a feature/capability you'd like to see, why not spend a couple of hours and help build it? The process is straight-forward. * Fork the smart mirror [git repository](https://github.com/evancohen/smart-mirror). * create a branch from the dev branch following the following naming convention `initials/feature-name` for example if Evan Cohen was creating a feature he would name the branch `ec/feature-name`. * Write the code for your feature/capability. * Create a Pull Request against the [**dev**](https://github.com/evancohen/smart-mirror/tree/dev) branch of the smart mirror. ## Development See the `dev` branch for features that are actively in development.\ If you would like to contribute please follow the [contribution guidelines](https://github.com/evancohen/smart-mirror/blob/master/CONTRIBUTING.md).\ To launch the mirror with a debug window attached use the following command: ``` npm start dev ``` ### Project Structure The smart mirror is an [Electron](http://electron.atom.io) app, which means it leverages Chromium, Node, and the V8 JavaScript engine to host and render the mirror. #### [app/](https://github.com/evancohen/smart-mirror/tree/master/app) The app directory contains the core of the smart-mirror: * `js/` core js of the smart mirror (bootstrapping angular, etc) * `css/` all of the smart mirror styles * `locales/` all of the localized speech commands (more on this later) * `fonts/` nobody knows what this folder does, but it's somehow necessary ;) #### [plugins/](https://github.com/evancohen/smart-mirror/tree/master/plugins) The plugins directory contains all of the plugins included in the smart mirror. A plugin consists of an optional combination of the following: * `config.schema.json` defines the configuration schema for your plugin. You can look at example schema and test your own at * `index.html` the html partial for your plugin. This will be added to the main [index.html](https://github.com/evancohen/smart-mirror/blob/master/index.html) as part of the new plugin location configuration. * `controller.js` All of the angular controller logic (data binding) for your plugin. This will be added to the main [index.html](https://github.com/evancohen/smart-mirror/blob/master/index.html) as part of the new plugin location configuration. * `service.js` should your plugin need an angular service this will be added to the main [index.html](https://github.com/evancohen/smart-mirror/blob/master/index.html) as part of the new plugin location configuration. * `plugin.css` plugin specific classes for styles * `locales\*.json` - this folder contains the plugin specific translations required for both configuration )the config.schema.json file and plugin runtime (index.html) , \* starting in V 0.28 * a complete sample plugin can be loaded from [here](https://github.com/sdetweil/samplePlugin) #### [remote/](https://github.com/evancohen/smart-mirror/tree/master/remote) The remote directory contains the code that powers the client configuration page(s). The "server" side code can be found in [`remote.js`](https://github.com/evancohen/smart-mirror/blob/master/remote.js). #### [app/locales/](https://github.com/evancohen/smart-mirror/tree/master/app/locales) Within this directory you will find the core `JSON` localization files for the mirror. The mirror uses [i18n](https://angular-translate.github.io/) to `$translate` strings rendered in the mirror (examples of this in `index.html` and `config.json`). When making changes to these files make sure that you add string keys to all localization files, not just to the one that you speak. #### [scripts/](https://github.com/evancohen/smart-mirror/tree/master/scripts) Contains various help scripts. There aren't a ton today and they will expand over time. ### Tips and tricks The dev console is your friend! You can set breakpoints, log messages, and inspect the DOM - just like you would in a regular webdev console. #### Speech Simulation Since speech query limitations have becoming a limeting factor of development I've created a shim for Annyang to "simulate" speech to test the mirror. In the dev console try some of these examples: ```javascript // Play YouTube video annyang.trigger("show me how to tie a bowtie"); // Play a song on SoundCloud annyang.trigger("SoundCloud play Kero One so seductive"); // Display a map annyang.trigger("show me a map of Seattle Washington"); ``` #### Dev Environment I typically wouldn't recommend developing directly on the Pi (unless you are trying to debug a Pi specific issue, in which case, I'm sorry. It's not the fastest thing in the world.) The mirror is compatible with linux and OSX, and developing there will be much nicer. You can plug your mirror into your computer and use it as a second (or third, you lucky duck) monitor. There's some logic in `main.js` that will automatically put the electron app on to your secondary monitor. It is not suggested that you develop on windows, the mirror is incompatible due to limitations of Snowboy. However, you can use a VM with ubuntu on it if you must... However, if u want to use Windows as a development environment, you can you can use the [WinScp](https://winscp.net/eng/index.php) or [bitvise](https://www.bitvise.com/ssh-client) ssh clients to access the PI disk over SSH (make sure to enable it on the pi) and bot tools provide a file manager type view over the PI disk, so you can double click edit with your favorite Windows editor.. For Linux and Mac you can do the same with their file managers or addons. Like [cyberduck for OSX](https://cyberduck.io/) or caja for linux # Updating Updating the mirror to get the latest updates is easy, just run: ``` git pull && npm install ``` This will sync the remote changes from whatever branch you are on (most likely `master`) to your local machine. This will also install/update any new or updated dependencies. #### Encountering errors when trying to run the mirror after an update? Sometimes new dependencies are added to the mirror or old ones are updated. In this case you need to update/install them: ``` npm update npm install ``` #### Still having problems? Occasionally there can be strange issues with the changes in dependencies. In this case the easiest thing to do would be to clean your repository. This will remove all uncommitted changes (except for your config file) and re-install all dependencies: ``` git clean -xdf -e config.json npm install ``` # Index * [Introduction](/) * [Hardware](/hardware) * [Installation](/installation) * [Install Raspberry PI OS](/installation/installing_raspbian) * [Install dependencies](/installation/install_dependencies) * [Configuration](/configuration) * [Configure the Pi](/configuration/configure_the_pi) * [Configuring Sound](/configuration/configuring-sound) * [Cloud Speech Recognition](/configuration/configuring_voice) * [First Time Running Smart Mirror](/configuration/first_time_running_smart_mirror) * [Configure the smart-mirror](/configuration/configure_the_mirror) * [Running](/running) * [Setting up Smart-Mirror to Run on Boot](/running/setting_up_smart-mirror_to_run_on_boot) * [Commands Used to Run Smart-Mirror](/running/commands_used_to_run_smart-mirror) * [Troubleshooting & FAQs](/troubleshooting) * [Issues installing electron-prebuilt](/troubleshooting/npm_install_issues) * [Microphone and Speech Recognition issues](/troubleshooting/microphone_and_speech_recognition_issues) * [Issues with Remote and ConfigUI](/troubleshooting/issues-with-remote-and-configui) * [Development and Contributing](/development_and_contributing) * [Updating](/updating) * [Index](/summary) * [How Tos](/how_tos) * [How To Obtain Chromium Speech Keys](/how_tos/how_to_obtain_chromium_speech_keys) * [How To Install Raspberry Pi OS(full)](/how_tos/how_to_install_raspbianfull) * [Enabling Motion Detection](/how_tos/enabling-motion-detection) # How Tos # How To Obtain Chromium Speech Keys ![](/files/taGx2x4h88gIzTF0Ysna) > ## `WARNING: This is no longer required` \#####It is being kept here for legacy reasons \##Enabling API Libraries 1. Make sure you are a member of [chromium-dev@chromium.org](https://groups.google.com/a/chromium.org/forum/?fromgroups#!forum/chromium-dev) (you can just [subscribe](https://groups.google.com/a/chromium.org/forum/?fromgroups#!forum/chromium-dev) to chromium-dev and choose not to receive mail). > For convenience, the APIs below are only visible to people subscribed to that group. 1. Make sure you are logged in with the Google account associated with the email address that you used to subscribe to chromium-dev. 2. Go to 3. Click the blue `Create Project` button. 4. (Optional and unlikely for this project) You may add other members of your organization or team on the Team tab. 5. In the `APIs & auth` >> `APIs tab` >> `API Library tab`, search for all of the following APIs. If you're a member of the chromeos-dev Google group you should see all of them. For each of these APIs click on them when found by the search, and then click on "Enable API" button at the top, read and agree to the Terms of Service that is shown, check the "I have read and agree to API name "Terms of Service" checkbox and click Accept: (This list might be out of date; try searching for APIs starting with "Chrome" or having "for Chrome" in the name.) * Calendar API * Contacts API * Drive API ***(Optional, enable this for Files.app on Chrome OS and SyncFileSystem API)*** * Chrome Remote Desktop API * Chrome Spelling API * Chrome Suggest API * Chrome Sync API * Chrome Translate Element * Chrome Web Store API * Chrome OS Hardware ID API ***(Optional, Chrome OS)*** * Device Registration API ***(Optional, Chrome OS)*** * Google Clound DNS API * Google Cloud Storage * Google Cloud Storage JSON API * Google Maps Geolocation API > > (requires enabling billing but is free to use; you can skip this one, in which case geolocation features of Chrome will not work). Entering your `geoPosition` in `config.js` will be required if you don't enable billing. * Google Maps Time Zone API * Google Now For Chrome API \*\*\*(Optional, enabled to show Google Now cards) * Google+ API * Nearby Messages API * Safe Browsing API * Speech API ***(NOT Google Cloud Speech API)*** * YouTube Data API ***(You can use the same API key for YouTube and might as well add that library now)*** > If any of these APIs are not shown, recheck step 1. \##Acquiring Keys 1. Go to the Credentials tab under the APIs & auth tab. * Click the "Add credentials" button then click on the "OAuth 2.0 client ID" item in the drop-down list. * Click on the "Configure consent screen" button. Fill in the "Product name" (name it anything you want) and other details if you have available then click on "Save" at the bottom. * Return to the Credentials tab and click the "Add credentials" button again, then select "OAuth 2.0 client ID" from the drop-down list. * In the "Application type" section check the "Other" option and give it a name in the "Name" text box, then click "Create" * In the pop-up window that appears you'll see a client ID and a "client secret" string. Copy and paste those in a text file on your dev box then click OK to dismiss it. * A new item should now appear in the "OAuth 2.0 client IDs" list. You can click on the name of your client id to retrieve the ID and secret at any time. In the next sections, we will refer to the values of the “Client ID” and “Client secret” fields. * Click the "Add credentials" button again on the same page. * In the pop-over window that shows up click the "Browser key" button. * Click on the "API key" item in the drop down list. * Fill in the name of the key or leave the default string there. * Leave the "Accept requests from these HTTP referrers (web sites) empty. * Click the "Create" button. * A pop-over should show up giving you the API key. Copy and paste it in a text file to save it, although you can access it later as well. * Click OK to dismiss this. You should now have an API key and a OAuth 2.0 client ID in on the Credentials tab. The next sections will refer to the value of the “API key” field too. > Note that the keys you have now acquired are not for distribution purposes and must not be shared with other users. # How To Install Raspberry Pi OS(full) \#####DOWNLOAD THE IMAGE * Official images for Raspbian Jessie are available to download from the Raspberry Pi website [Downloads page.](https://www.raspberrypi.org/downloads/raspbian/) * Do not use Raspbian Lite or NOOBs to install Raspbian Jessie. This causes unexpected results when installing and configuring your Smart-mirror * After downloading the .zip file, unzip it to get the image file (.img) for writing to your SD card. \#####WRITING AN IMAGE TO THE SD CARD * With the image file of the distribution of your choice, you need to use an image writing tool to install it on your SD card. * See Raspberry Pi's guide for your system: * [Linux](https://www.raspberrypi.org/documentation/installation/installing-images/linux.md) * [Mac OS](https://www.raspberrypi.org/documentation/installation/installing-images/mac.md) * [Windows](https://www.raspberrypi.org/documentation/installation/installing-images/windows.md) # Enabling Motion Detection \#Enabling Motion Detection ![](/files/taGx2x4h88gIzTF0Ysna) > ## `WARNING: Only compatible with Raspberry Pi` \#####Do not enable or attempt to install on any thing other than a Smart-Mirror compatible Raspberry Pi Device. #### Installing Dependencies Motion Detection requires Johnny-five.io as well as Raspi-io. make sure you're within the smart-mirror folder. ```bash cd ~/smart-mirror ``` from the smart-mirror folder run the following command. ```bash npm install johnny-five && npm install raspi-io ``` #### Configuration Use the Motion Settings of the ConfigUI to configure and enable the motion detection after installing dependencies. | Variable | Usage | Data Type | Default Value if not included in config.js | | -------- | ------------------------------------------------------------------------------------------------------- | --------- | ------------------------------------------ | | pin | Identify GPIO input Pin connected to output pin of the PIR device or other device used to detect motion | string | GPIO26 | | enabled | enable motion detection | boolean | false | #### Making it all work **Parts required** * [PIR Device](https://smile.amazon.com/gp/product/B00FDPO9B8/ref=oh_aui_search_detailpage?ie=UTF8\&psc=1) * (optional!) LED (color of your choice) * (optional!) Resistor (based on draw of LED) **Wiring Diagram with LED** ![figure 1](/files/sMaPIGdD3AvX5YnFDQj0)`[figure 1]` **Wiring Diagram without LED** ![figure 2](/files/UljVxeCuxeIUHKoncnPw) `[figure 2]` #### Basic Functionality Motion detection works with AutoTimer Settings. Using Johnny-Five's motion API the PIR device is connected to a PIN on the Raspberry Pi. Suggested PIN is GPIO 26 as illustrated in figure 1 and figure 2 above. When the `motionstart` event is triggered the `auto-sleep timer` is stopped and will remain stopped until the `motionend` event is triggered. When the `motionend` event is triggered the `auto-sleep timer` is started and set to the `config.autoTimer.autoSleep` interval set in configUI. #### Issues A live chat to get help and discuss mirror related issues: [discord chat](https://discord.gg/EMb4ynW). Usually there are a few folks hanging around in the lobby, but if there aren't you are probably better off [filing an issue](https://github.com/evancohen/smart-mirror/issues/new). Please tag @justbill2020 on any motion detection issues.