Exception has occurred: KeyError 'el' File "example2.py", line 12, in while 0 == session.read(): KeyError: 'el'
I’m not sure if this is a problem unique to my GPS receiver, but it regularly sends data that is not “class”:”TPV” (eg: “class”:”SKY”) – TPV is the data with coordinates, and whenever that happens, all the sample code throws a KeyError.
I’ve found the easiest way to fix this is to add a TRY/EXCEPT inside the forever loop, catch the KeyError expection, and PASS. Here is an adaptation of code I found on Stackoverflow that solves the problem – I’ve submitted it as an improvement, hopefully they accept my edits!
import threading
import time
from gps import *
from datetime import datetime
class GpsPoller(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self.session = gps(mode=WATCH_ENABLE)
self.current_value = None
def get_current_value(self):
return self.current_value
def run(self):
while True: #moved the try INTO the forever loop, so pass from the except resumes collecting data
try:
self.current_value = self.session.next()
time.sleep(0.2) # tune this, you might not get values that quickly
except KeyError as e: #catch the KerError, and return to the forever loop
pass
except StopIteration:
pass
#if __name__ == '__main__':
gpsp = GpsPoller()
gpsp.start()
# gpsp now polls every .2 seconds for new data, storing it in self.current_value
while 1:
# In the main thread, every 5 seconds print the current value
time.sleep(5)
value=gpsp.get_current_value()
print(value)
Update (May 6, 2023): I found a better solution: the gpsd-py3 library works great.
When I presented my blogging bot here a couple months ago, a friend suggested I try to create generated video content – check out the linked video to see what I’ve been able to do so far: https://youtu.be/WkfGq42OyBI
It’s not quite there – the body movements are random, the eyes aren’t focused on the camera, some of the mouth movements don’t seem to match the face. I’ve added a street scene in the background to make it a bit more interesting.
Like the blogging bot, the script is generated by an OpenAI model, seeded with comments scrapped from an internet forum. The audio is created with a text to speech tool from ElevenLabs. I experimented with a few different tools for creating an animated avatar, and settled on working with Nvidia’s Audio2Face animation model in combination with Metahuman technology in Unreal Engine, an environment popular in game development.
Unlike my blogging bot, the process for creating this video is not automated. The tooling, at least for someone with my experience and resources, does not seem to lend itself to automation. It looks like this could change in the very near future – Nvidia has announced, but has not yet released, its Omniverse Avatar Cloud Engine (ACE), which looks like it could facilitate the creation of generated content. If anyone from Nvidia is reading – I’d love early access.
The Guardian reported earlier this week that Kuwait News has introduced a generated news presenting Avatar. I could envision services like LinkedIn experimenting with using an avatar to present our personal news feed as generated video. It remains to be seen if this new generation of avatars will see greater success than earlier attempts like Microsoft Office’s Clippy!
In January, I created a blogging bot as a sandbox for playing with generated content and tooling. The bot identifies a popular post on an internet forum, collects some of the comments, submits them to ChatGPT to write an article, and then posts it to the web, Fediverse, and Twitter. Read about it here https://articles.hotelexistence.ca/posts/bloggingbot/ , and then check out the bot at https://www.eliza-ng.me/ . Source code: https://github.com/raudette/bloggingbot. It reached an interesting milestone this week – one of its posts was organically starred on Mastodon.
In November, I was following a news story the Washington Post broke about Trustcor, a Certificate Authority with root certificates pre-loaded in all popular browsers. A few academics identified some problematic relationships and facts about Trustcor that puts into question the process used by Chrome, Firefox, and Safari to determine which companies can issue the certificates that our browsers use to indicate a connection is secure. From the article:
The website site lists a contact phone number in Panama, which has been disconnected, and one in Toronto, where a message had not been returned after more than a week. The email contact form on the site doesn’t work. The physical address in Toronto given in its auditor’s report, 371 Front St. West, houses a UPS Store mail drop.
Would this company pass your audit? I was familiar with the role played by root certificates in browsers, and understood there was an assessment process for inclusion completed by the teams that build browsers. I get that a process can’t catch everything, but I was surprised about how flagrant this evidence of untrustworthy behaviour was.
I was pleased to see in the Post’s follow up story that Mozilla and Microsoft are removing Trustcor’s root cert from their browsers. This follow-up article also presents the possibility that the root cert was leveraged by Packet Forensics for man-in-the-middle interception of encrypted traffic:
Packet Forensics first drew attention from privacy advocates a dozen years ago.
In 2010, researcher Chris Soghoian attended an invitation-only industry conference nicknamed the Wiretapper’s Ball and obtained a Packet Forensics brochure aimed at law enforcement and intelligence agency customers.
The brochure was for a piece of hardware to help buyers read web traffic that parties thought was secure. But it wasn’t.
“IP communication dictates the need to examine encrypted traffic at will,” the brochure read, according to a report in Wired. “Your investigative staff will collect its best evidence while users are lulled into a false sense of security afforded by web, email or VOIP encryption,” the brochure added.
Researchers thought at the time that the most likely way the box was being used was with a certificate issued by an authority for money or under a court order that would guarantee the authenticity of an impostor communications site.
They did not conclude that an entire certificate authority itself might be compromised.
Can you imagine the potential for man-in-the-middle abuse by ISPs and VPN service providers if they had root certificates in our browsers (examples of untrustworthy behaviour by VPN providers)? Given the challenges with this model, it’s interesting we haven’t seen more evidence of widespread abuse:
Was this situation isolated and unique?
Why doesn’t every malicious actor get a root certificate?
Are there limited incentives to intercepting traffic?
Perhaps regular law/ethics are sufficient to deter intercepting traffic?
Is this happening everywhere all the time, and there’s limited impact?
Perhaps most traffic isn’t of value, so traffic interception is limited and targeted?
I recommend reading Delegating trust is really, really, really hard, a reflection on the matter by Cory Doctorow. Hopefully, the Trustcor story brings greater attention to the issue, and more attention will be paid by development teams & users to the root certificates pre-loaded in their browsers.
“koala bear eating eggs benedict in a bistro” is the prompt I entered into OpenAI’s DALL·E system to generate this image. I have been reading articles about AI image generation since DALL·E 2 launched earlier this year, and have been experimenting with it hands-on since I received access earlier this week. It is lots of fun, but doesn’t always generate the results you might expect. I’ve been trying to describe to it artist Henri Julien’s Chasse-gallerie, a drawing of 8 voyageurs flying in a canoe at night, and DALL·E struggles with the flying canoe. For each prompt, DALL·E initially creates 4 image variations – I’ve selected the most interesting one for each below.
La Chasse-galerie by Henri Julien4 men in a flying canoe at night, generated by DALL·E 4 men paddling a flying canoe through the sky at night, generated by DALL·E4 men in a canoe in the sky at night, generated by DALL·E lumberjacks flying in a canoe past the moon, by DALL·E. I like how they are chopping the tree while flying the canoe.
Generating Images on your PC
There are models that exist that you can run on your home PC. I’ve checked out min-dalle (used by https://www.craiyon.com/) and stable-diffusion (demo). They can both create imagery that roughly matches my prompts, which is amazing. I found that the output from min-dalle was a bit crude, with distorted features. Output I’ve generated from stable-diffusion is much better.
For me, quality really makes a difference in how much I get out of the tool. Generating low quality images isn’t as much fun. Perhaps this is a reflection of my artistic skills – I can make a crappy drawing of a panda climbing with a ball point pen while watching a Powerpoint in a random meeting. But I personally don’t have the skill to render my ideas as well as DALL·E – maybe that’s what makes it so fascinating.
A painting of a panda climbing a skyscraper, generated by DALL·E miniA painting of a panda climbing a skyscraper, generated by stable-diffusionA painting of a panda climbing a skyscraper, generated by DALL·E
Not all fun and games
As much fun as this is, there is a lot of controversy about the implications of making tools like this accessible.
NBC News actually has a pretty good article about the biases exhibited by DALL·E. For example, ‘a flight attendant’ only produces images of women. In this respect, it is very much like other tools currently in the marketplace – I was curious, and searched Getty Images for ‘flight attendant’ stock photos, and found only women in the top results. Image generation tools continue to propagate the bias problems we see everywhere.
An Atlantic columnist raises some other interesting points about the backlash he faced when he used generated images in his newsletter, as opposed to professional illustration that you might see in a magazine feature. Here are some further thoughts on the ideas he presented:
Using a computer program to illustrate stories takes away work that would go to a paid artist, “AI art does seem like a thing that will devalue art in the long run” – I almost wonder here, if the computer program just becomes another tool in the artist’s arsenal. Is the value of an illustrator in the mechanics of creating an image, or visually conveying an idea? What if we think of a program like DALL-E as a creative tool, like Photoshop. Did Photoshop devalue photography?
There is no compensation or disclosure to the artists who created the imagery used to train these art tools – “DALL-E is trained on the creative work of countless artists, and so there’s a legitimate argument to be made that it is essentially laundering human creativity in some way for commercial product.” – At primary school age, my children brought home art created using pointillism techniques, without compensating or disclosing inspiration from Georges Seurat and Paul Signac. Popular image editing packages have had Van Gogh filters for years. We all learn and build on the work of those who preceded us. Once a style or idea is presented to the world, who owns it? Is the use case of training an algorithm a separate right, different from training an artist?
There are also challenges with image generation tools facilitating the creation of offensive imagery and disinformation, making it easier to cause harm than it is today with existing tools like Photoshop.
These tools will continue to progress, and will create change in ways, and on a scale that are hard to predict. It remains to be seen if we collectively decide to put new controls in place to address these challenges. In the meantime, I’ll be generating imagery of pandas and koalas in urban environments.
I have been in IT “war room” type situations a number of times, working to get service to a production system restored. It was with professional interest that I followed the Rogers outage on July 8th, 2022.
For anyone looking for more information than what was covered in the media, Rogers’ response to the CRTC’s questions about the incident was published and can be downloaded from the CRTC (the .DOCX link on the July 22, 2022 post).
Rogers response has been redacted, and is light on specifics (eg: no information about their network). However, there were some interesting details, such as how Rogers issues alternate mobile SIMs from competing mobile carriers to some its technical teams to maintain contact in the event of an outage like this one.
I recently participated in GeekWeek 7.5, the Canadian Centre for Cyber Security’s (CCCS) annual cybersecurity workshop. I was assigned to a team of peers in banking, telecom, government and academia. We were to work together on analyzing how Bluetooth item trackers (eg: Apple AirTags, Tile) can be covertly used for malicious purposes, and developing processes and tools to detect them.
It was my first time attending the event, and I wasn’t sure what to expect. Here’s how it worked. The CCCS builds teams from the pool of GeekWeek applicants, based on interests and skills identified by the applicant in the application process. Leading up to the event, CCCS appoints a team lead, who defines goals for the team.
Testing a homemade AirTag clone
I was appointed as a team co-lead and assigned a team. My co-lead and I were to define a challenge in the IoT space. Inspired by recent headlines concerning stalking facilitated by AirTags, my co-lead suggested analyzing how item trackers can be covertly used for malicious purposes, and developing processes and tools to detect them.
“Can you find my stealth AirTag clone in this data?” we asked the data scientist on our team. 🍰 <piece of cake>, he responded.
As various news outlets reported about the malicious use of these trackers, manufacturers implemented anti-stalking measures which were quickly defeated by research efforts such as find-you. The find-you tracker avoids detection by rotating through identifiers – to an iPhone, it appears no differently from walking past other people with iPhones.
The team built find-you stealth trackers which rotated through 4 keys and went to work. We wanted to see if we could develop a technique for detecting and identifying the trackers. Data was collected by stalking ourselves, walking through busy urban areas with find-you trackers, and logging all Apple Find My Bluetooth advertisements.
Our hypothesis was that we could identify the find-you tracker based on signal strength patterns. We believed that the signal strength of the find-you tracker we were wearing would be consistent over time, as its location relative to our data logger wouldn’t change, and other Find My devices would vary, as we walked past other pedestrians carrying devices.
Rotates keys on a random interval, between 30 and 60 seconds (Find-You rotates every 30 seconds)
Transmits at random power levels, in an attempt to avoid detection based on a consistent signal strength.
In our testing, even with these changes, we were still able to identify stealth tags in an area with other Apple Find My devices. Even at the ESP32’s lowest power level, the stealth tracker can be identified as the signal strength of the stealth tracker is higher than other devices the vicinity of the tracked subject.
Further areas for exploration would be reducing signal strength with shielding material or some other means, and adding an accelerometer such that the device only transmitted Bluetooth advertisements when the target subject is in motion.
Building a Clone Tile Tracker
The team was inspired by Seemoo Lab‘s work on the AirTag. Could we do similar things with the Tile tracker?
And, we figured out some things, but got stumped. Tile uses the Bluetooth MAC address to track a Tile. We got our clone Tile to the point where we could take MAC of a tile registered to our account, load our firmware with the MAC onto an ESP32, and walk around with our Tile app and it would track it.
But, it seemed like if we sent our Tile’s MAC ID to someone else to load on to their ESP32, and track with their Tile app, it wouldn’t report the location. And, although Tile reports thousands of users in my area, even a genuine Tile didn’t seem to get picked up by other Tile users walking through office food courts, malls, or transit hubs. As a result, at the end of two weeks, many questions remained unanswered.
The experience of stepping away from the day-to-day, learning about different technologies and building things was very similar to my previous experiences participating in corporate hackathons.
What made GeekWeek exciting for me was leading a team of people from different companies, with different professional backgrounds. At a corporate hackathon, everyone already knows their teammates, what they’re capable of, they share the same corporate culture, they typically work on the same software stack, and have the same collaboration tools pre-loaded on their laptops. At GeekWeek, it was interesting just breaking down the problem, and finding out who had what tools, what experience, and was best suited to take on a task. Also, it was interesting to hear about everyone’s professional work – some were really pushing boundaries in their fields.
Ever since I first read about Hackinthoshes, I’ve thought about building one. A friend of mine edits all of his video on a purpose built Hackintosh. I never did build one – for myself, I like to run Linux, I don’t really need a Mac for anything, and I find that off-lease corporate grade laptops are the best value in computing. But, every once in a while, I have something I want to build on my iPhone, and a Mac is like a dongle that makes it possible.
Simple things, like finding out why the site I’ve built with the HTML5 geolocation API will work on my PC, but not on my iPhone, are just not possible without a Mac. To solve these types of problems in the past, I’ve just borrowed a Mac from the office.
Recently, another project came up, and I decided to try to build a virtual Hackintosh with OSX-KVM – a package that simplifies running OSX in a virtual machine. Years ago, I tried running OSX in VMWare Player – this would have been on a Intel Core 2 system at the time – in my opinion, it was interesting, but unusable. My experience with OSX-KVM has been much better.
Just following the instructions, within a couple of hours, I had OSX Big Sur running in a VM. OSX-KVM does almost everything, including downloading the OS from Apple. My PC is pretty basic – a Ryzen 3400g, SSD, 16 GB. I assigned the VM a couple CPU cores and 12 GB of ram, and it’s usable.
In a few hours of dabbling around, I’ve come across four issues:
First, I couldn’t “pass through” my iPhone from the host PC to the guest OSX OS. The best overview of this issue I could find is logged on Github. There are configuration related workarounds, but I decided the easiest way to solve it was to acquire a PCIe USB controller and allocate it to the VM. PCIe cards with the FL1100 and AS3142 chipsets are said to work with OSX – I ended up buying this AS3142 card, as it was the one I could get my hands on quickest, and it seems to work fine – I can see my iPhone in OSX now, and run the dev tools for Safari iOS.
Second, I can’t log into Apple ID, and as a result, I can’t compile iOS apps in Xcode. It looks like this is solvable.
Chrome doesn’t display properly. I wonder if this is related to graphics acceleration. I don’t need Chrome to work, but its a reminder to me that this exercise is really just a big hack – I don’t think I would count on this to do any real work.
Finally, I seem to lose control keyboard/mouse of the VM if it goes to sleep, and I can’t seem to get it back. I’m sure this is solvable, but addressed it by turning sleep mode off.
Presumably, Hackintoshes will eventually stop working as Apple moves its platform to ARM processors, but for now, it’s definitely worth trying out.
Update (2022/03/03): The Apple ID issue was addressed by following the previously linked instructions, the Chrome issue was resolved by turning off GPU acceleration (–disable-gpu).
Update (2022/03/26): I needed Bluetooth. I bought an Asus BT-400 adapter, it seems to work fine. AirDrop functionality doesn’t seem to work – haven’t investigated. And – not a Hackinstosh issue, but a Mac issue – the CH9102F serial chip on one of my ESP32 microcontroller boards isn’t natively supported by Big Sur, found a third-party driver. Apple Maps doesn’t work, as it requires a GPU, and the OpenHaystack project that I’m playing with requires Maps.
I wanted to build my own vision model for a few reasons:
I wanted to learn how
In my limited experience with OpenALPR, it looked like it was missing some license plates that seemed fairly readable to my eyes – could I possibly do better training my own model?
Just the way it is built, I know I wouldn’t be able to get OpenALPR to run faster on my Pi – I wouldn’t be able to get it to run faster by off loading image processing to a VPU like the Myriad X in my Oak D camera.
The gen2-license-plate-recognition example provided by Luxonis, built from Intel’s Model Zoo, does not work well with Ontario license plates
The first step was building a library of images to train a model with. I sorted through hundreds of images I’d taken on rides in September, and selected 65 where the photos were clear, and there were license plates in the frame. As this was my first attempt, I wasn’t going to worry about sub-optimal situations (out of focus, low light, over exposed, etc…). I then had to annotate the images – draw boxes around the license plates in the photos, and “tag” them as plates. I looked at a couple tools – I started with Microsoft’s VoTT, but ended up using labelimg. Labelimg was efficient, with great keyboard shortcuts, and used the mouse scroll wheel to control zoom, which was great for labeling small details in larger photos.
I then tried one tutorial after another, and struggled to get them to work. Many examples were setup to run on Google Colab. I found when I was following these instructions, and I got to part where I was actually training the model, Colab would time out. Colab is only intended for short interactive sessions – perhaps it wouldn’t work for me as I was working with higher resolution images, which would take more computing time.
What I ended up doing was manually running the steps in the Train_YoloV3.ipynb notebook from pysource, straight into the console. As my home PCs don’t have dedicated GPUs, I setup a p3.2xlarge Amazon EC2 instance to run the training. If memory serves, training against those 65 images, using the settings from that tutorial, took a couple of hours.
I took the model I created from my September rides, and then tested it against images from my October rides – I’m surprised how well it worked.
My Yolov3 model running on Oak-D
Since training that model, I’ve been on the lookout for an nVidia video card I can use for training at home. It’s hard to know for sure, but it seems it wouldn’t take long to recoup the cost of a GPU vs training on an EC2 instance in the cloud, and I can always resell a GPU. I’ve tried a few times with the fastest CPU I have in the house (a Ryzen 3400g), and it just doesn’t seem feasible. I haven’t seen a cheap GPU option, and the prices just seem to be going higher since I started looking in November.
I don’t have usable code or a useful model to share at this point, at this point, I’m mostly learning and trying to figure out the process.
I wanted a hard copy of an eBook I had that is out of print. There are many resources out there for binding books. Many recommend using acid free PVA glue. I can’t speak to how it compares to other glues, but “Aleene’s Tacky Glue” is a PVA glue, available acid free, which was available at craft stores in my area.
This post will focus on prepping an eBook for print. As US Letter is the common paper size here, which is too big for a book, I decided to print 4 pages per US letter page, 2 pages per side, each 5.5″ wide by 8.5″ tall.
First, I loaded to book into Calibre, opened it, and printed it to PDF. For this exercise, I’ve used Ian Fleming’s Casino Royale, which is out of copyright in Canada.
Calibre Print to PDF
Next, I had to re-arrange the pages. If we just print 2 pages per side, duplex, page 4 will end up on the back of page 1. We want page 2 on the back of page 1 – we want to reorder the PDF following the patterns 1, 3, 4, 2, 5, 7, 8, 6… This LibreOffice spreadsheet might help: Pages.ods
Pages have to be re-ordered for regular duplex printing – page 2 has to be on the back of page!
Next, I used a tool called pdfjam to fit 2 pages per side: pdfjam collated.pdf -o collated-2perpagealternate.pdf --nup 2x1 --landscape
I sent this PDF to my local printer, and had them cut the pages in half for me. With this output, I bound the book, roughly following a Youtube tutorial. My book turned out OK, but it feels like it would take me a few more attempts to get a book as sturdy as a commercially bound book.
My personal brain dump, Opinions, Projects, Toronto
