All posts by raudette

project

Smart Dashcam for Bicycles – Part 5: Blindspot Detection

1 Comment

I continue to experiment with how a dashcam can assist urban cyclists. This time, I’ve started a fresh design with a different idea, a new camera, new models, and new code, which I’m submitting as an entry for the Toronto ♥️’s Bikes Make-a-Thon.

I enjoy biking from my home in North York, near Mel Lastman Square, to my office near Union Station during the week. The most harrowing part of this ride is the Yonge-401 interchange, which requires two lane changes with fast moving traffic from the 401 on and off ramp.

Yonge - Hwy 401 Interchange
Yonge – Hwy 401 Interchange

As cyclists in the city, we all have “scary spots” like these on our routes. I would like to present you with a Smart Dashcam for Bicycles as a tool for these challenges. A dashcam could:

  • increase safety
  • collect evidence in the event of an incident
  • gather data

For the purposes of the Make-a-Thon, I have built a smart dashcam with blind spot detection, similar to what you would see in modern cars. The IIHS says that in cars, this feature lowers the rate of all lane-change crashes by 14 percent.

My prototype consists of a laptop, a USB AI accelerated camera from Luxonis mounted to my bicycle seat post, and my smartphone as a display. It’s a few hundred lines of Python code that builds on a freely available AI vehicle recognition model from the Intel Open Model Zoo. I’ve built on the license plate recognition and MJPEG video streaming sample code from Luxonis that was supplied with the OAK-D camera. I tether the laptop to the smartphone using wifi, and I use an iOS app called IPCams to view the video stream.

Bicycle Dashcam with Smartphone Display
Bicycle Dashcam with Smartphone Display

The vehicles are recognized and identified. The video is streamed over wifi to the smartphone. A caution alert is added to the video when a vehicle is detected.

Phone screen shot. Caution, car approaching
Phone screen shot. Caution, car approaching

A demo video can be found here: https://youtu.be/zMTRDsA6uJM

In this proof of concept, the dashcam is just a fancy, complicated, expensive, rear view mirror. A final version would expand on this functionality by integrating features such as:

  • Sounding an audible alert when danger is detected
  • Recording the speed and proximity of the cars around you
  • Integrated GPS
  • Cloud and social features for sharing data with the city and fellow cyclists
  • A car driver readable display, eg: “Driver ABCD1234, your current speed is 45”. Like a mobile Toronto Watch Your Speed program sign. Would a driver allow a cyclist more space if they were aware their actions are being logged?

Parts:
Luxonis OAK-D Camera
Bike Phone Mount
Bike Camera Mount
IPCams for iOS (to watch MJPEG stream)

Source:
https://github.com/raudette/SmartDashcamForBikes

Previous Articles:
See my previous articles on bicycle dashcams:
Part 4, Part 3, Part 2, Part 1

UPDATE: This project was featured on Hackaday, November 1, 2021.

computing, security, thoughts

Mesh network in my neighborhood. Not how I had envisioned it.

Leave a comment

I got an email from Amazon today. I’m automatically opted-in to “Amazon Sidewalk”, unless I choose to opt out. Amazon Sidewalk allows devices participating their Sidewalk program to connect to the Internet through Amazon devices, like the Amazon Echo.

Amazon Sidewalk – a mesh network for Amazon devices to connect to the Internet:
https://www.amazon.com/Amazon-Sidewalk/b?ie=UTF8&node=21328123011

Not exactly how I’d envisioned a neighborhood mesh, but their “read the fine print to opt out” strategy will probably work better than my asking neighbors to build a network.

computing, project

1-Click Passwords

Leave a comment

I was recently presented with a situation where I would have to regularly enter a 48 random character password for a month or more to log in to a computer that was assigned to me. Given that I couldn’t possibly memorize this string, and the computer is reasonably physically secure, I decided to build a device to do this for me.

I had previously used an Arduino to emulate a gamepad for a homemade Dance Dance Revolution mat. This time, I needed to emulate a keyboard. A search for “HID Arduino” returned the Arduino HID page, which suggested an Arduino with an Atmel 32u4 microcontroller. A search for Arduino 32u4 on Amazon returned the KeeYees Pro Micro clone, which I ordered.

Arduino Pro Micro Clone, button wired to I/O 4

It came in, I soldered a button to I/O 4, and uploaded the following code:

include "Keyboard.h"
include "Bounce2.h"
const int buttonPin = 4;
Bounce bounceTrigger = Bounce();
void setup() {
bounceTrigger.attach(buttonPin, INPUT_PULLUP );
Keyboard.begin();
}
void loop() {
bounceTrigger.update();
if ( bounceTrigger.rose() ) {
Keyboard.println("I put my password here");
}
}

Now, every morning, instead of copying 48 characters from a Post-it, I just click the button.

It should be said, this defeats the purpose of the password, and the password isn’t stored in a secure way on the microcontroller. But this technique is great for any time you need to automate a sequence of keystrokes.

project

Bicycle Dashcam Part 4: New Hardware

Leave a comment

I was reading an article about Oak Vision Modules on Hackaday, and thought, wow, this is the PERFECT platform for my bicycle dashcam. The Oak Vision module is a Kickstarter project with camera modules, depth mapping capability using stereo vision, and a processor (Intel Movidius Myriad X) designed to accelerate machine vision in 1 package for $149US – see https://www.kickstarter.com/projects/opencv/opencv-ai-kit/

At the 3:55 mark in the marketing video, I THEN see the board mounted to a bicycle saddle, which is EXACTLY what I want to do:

Luxonis OAK-D vision module prototype as seen in their Kickstarter video

I went to see what I could find about the developers, and read about them on TechCrunch:
“The actual device and onboard AI were created by Luxonis, which previously created the CommuteGuardian, a sort of smart brake light for bikes that tracks objects in real time so it can warn the rider. The team couldn’t find any hardware that fit the bill so they made their own, and then collaborated with OpenCV to make the OAK series as a follow-up.”

This is pretty exciting – CommuteGuardian is the first project I’ve come across with similar goals to mine: Prevent and Deter Car-Bicycle accidents. I exchanged a few emails with Brandon Gilles, the Luxonis CEO, and he shared some background – they also checked out OpenALPR, and started work on mobile phone implementations, but decided to move to a custom board when the Myriad X processor was launched.

You can read more about CommuteGuardian here:
https://luxonis.com/commuteguardian and
https://discuss.luxonis.com/d/8-it-works-working-prototype-of-commute-guardian

I decided to back Luxonis’ Oak project. I’ll have to learn some new tools, but this board will be much faster than the Pi for image analysis (much faster than the 1 frame per 8 seconds I’m getting now!). The stereo vision capabilities on the Oak-D will allow for depth mapping, a capability for which I had previously been considering adding a LIDAR sensor. Looking forward to receiving my Oak-D, hopefully in December. In the interim, I’ll continue to experiment with different license plate recognition systems, read more about the tooling I can use with Oak-D, and perhaps try a different camera module on the Pi.

project

Bicycle Dashcam Part 3: More field testing

Leave a comment

On a sunny mid-June Saturday, I took my bike for a ride down Yonge St to lake Ontario with my bicycle dashcam, testing my latest changes (May 18th). Over the course of a 2 hour ride, taking a photo about every 10 seconds:

  • Reviewing the photos with my own eyes, I can make out about 45 images with readable plates (not every image was usable or had a car in the photo)
  • Of these 45, OpenALPR can make out about 10
Sign misidentified as license plate
OpenALPR picked out this dry cleaning sign as license plate 2 H0UR

I’m going to try running these photos through alternate ALPR engines, and compare results.

On this run, I tested the Pi Camera V2’s various sensor modes: the streaming modes at 1920×1080 30 fps, 3280×2464 15 fps, 640×922 30 fps, 1640×922 40 fps, 1280×720 41 fps, 1280×720 60 fps, as well as the still mode at 3280×2464. Further testing is likely still required, but I continue to get the best results from the still mode – all of the successful matches were shot using still mode.

Dashcam Successfully Recognizes Plate
Dashcam Successfully Recognizes License Plate

I’m getting better results than I had on previous runs as a result of tweaking the pi-camera-connect NodeJS library to:

  • use a 5 second capture delay, which allows exposure time, gain, and white balance to be determined
  • set the exposure mode to sports, which reduces motion blur by preferentially increasing gain rather than exposure time

However, the images are still not as good as I would like.

Rolling shutter issues are apparent in some photos taken by the Pi Camera V2 while in motion

The Plate Recognizer service has an excellent article on Camera Setup for ALPR. It highlights many of the challenges I’m seeing with my setup:

  • Angle, lighting
  • 8 MP is suggested for highway or street monitoring – the Pi Camera is sufficient in this regard
  • Zoom – I think this is a challenge in my setup – I liked the idea of getting everything around me, but I think I have to reduce the area I capture to get a view of the plate with more detail. Perhaps focus to my “7 o’clock” rather than capture everything behind me.
  • At 30 mph (~45 km/hr), which probably covers most bike riding in traffic, they suggest at least 3 to 5 frames at 15-25 frames per second.

I might order the latest Pi camera with the zoom lens and see if I get better results.

Reviewing the data from my ride, there’s also an issue with my code that pulls the GPS coordinates from the phone, which I didn’t see when testing at home. I figure this is either the phone locking while I ride, and not running the javascript – I’ll try using the NoSleep.js library before my next test run.

project

Bicycle Dashcam Part 2: Camera upgrade + GPS

Leave a comment

In February, I saw Robert Lucian’s Raspberry Pi based license plate reader project on Hackaday.  His project is different, in that he wrote his own license plate recognition algorithm, which runs in the cloud – the Pi feeds the images to the cloud for processing.  He had great results – 30 frames per second, with 1 second of latency.  This is awesome, but I want to process on the device – I want to avoid cellular data and cloud charges.  Once I get this working, I’ll look at improving performance with a more capable processor, like the nVidia Jetson or the Intel Neural compute stick.

In any case, Robert was getting great images from his Pi – so I asked him how he did it.  He wrote me back with a few suggestions – he is using the Pi camera in stream mode 5 at 30 fps.

I wondered if one of the issues was my Pi Camera (v1), so I ordered a version 2 camera (just weeks before the HQ camera came out!).  The images I was getting still weren’t great.  I’m using the pi-camera-connect​ package, here’s what I’ve learned so far:

  • The best documentation I’ve seen for the Pi camera can be found at https://picamera.readthedocs.io/en/release-1.13/
  • ​Some of the modes are capable of higher frames per second, but results may be poor.  Start with 30 fps
  • In stream mode, streamCamera.startCapture() must be called 2-3 seconds before streamCamera.takeImage()
  • There are a number of parameters not exposed by the pi-camera-connect package, but ultimately, this package is just a front end for raspistill and raspivid.  All the settings can be tweaked in the source.  Specifically, ​​increase the –timeout delay for still images.  I also want to experiment with the –exposure sports setting.​

I still have more tweaking to do with the Pi Camera 2.  If after a few more runs, I don’t get the images I need, I’ll try the HQ camera or try interfacing with an action camera.

Finally, I’ve added GPS functionality.  If you access the application with your phone while you’re riding, the application will associate your phone’s GPS coordinates with each capture.​

Source: https://github.com/raudette/plates
Bicycle Dashcam Part 1

computing, project

iOS Safari’s WebSockets implementation doesn’t work with self signed certs

Leave a comment

I’m building a Node application hosted on a Raspberry Pi, that will not be connected to the internet. A user will interface with the application through the browser on their phone. The application calls the browser for its GPS coordinates using the HTML Geolocation API.

In iOS, the HTML Geolocation API only works for HTTPS sites. I found an excellent post on Stackoverflow for creating a self signed cert that works in most browsers. I created the cert, added it to my desktop and phone. HTTPS worked great.

I first tried the Node ws websocket library, and the Node application would call out to the browser to fetch GPS coordinates when it needed them.

The application worked great in Firefox and Chrome, but it would not work in the iOS browser. If I dropped to HTTP (vs HTTPS) and WS (vs WSS), it worked fine. For some reason, the iOS browser accepted the cert for HTTPS, but not WSS. Unfortunately, I needed HTTPS to use Geolocation.

I couldn’t get it to work. I ended up moving my application to Socket.IO, which has a fallback method to HTTPS polling if a websocket connection cannot be established. This worked for my scenario. If you need a websocket like capability and have to use self signed certificates on iOS, try Socket.IO.