Org and Groceries
One of the things that was interesting to me about Org-mode is how people from myriad vocations find it useful in (if not integral to) their day-to-day workflow. Unfortunately, I'm somewhat locked into my professional tool set, so a couple of months ago, I decided to try to use Org-mode to make my personal life more efficient in some way. This post marks my first foray into Org-ing up.
If there's one task that I do not like, it's going to the grocery store. I am scatterbrained, I admit it. I spend so much time wandering down aisles multiple times trying to get everything I'm after. My spouse also hates the grocery store, so it seemed like a good idea to subscribe to a meal delivery service. The recipes were very good for the most part. The only hard part was that we have some dietary restrictions that aren't reasonable for a meal subscription service to abide, but that was okay because we could make substitutions where necessary. We don't like to throw food away, so we naturally began to amass large amounts of the ingredients we were substituting for. About six months into our subscription, we had forty-something recipes and I decided it was time to end the subscription. Of course, this meant I was going to take care of the groceries and cooking and hopefully Org-mode would help me manage it.
My first task was to get my recipes to Org-mode. My subscription service provides recipes on letter-sized cards. There's a small time frame where you can get the recipes online, so scraping HTML was a non-start. I figured my best option for a programmatic approach was to use OpenCV to scrape the text from the recipe cards; each chunk of text on a card is laid out consistently, so it seemed like writing a scraper in Python should be easy. After scanning all the recipes to images at my local library, I picked one and got to work converting the OpenCV output to an Org heading. I probably spent an hour or so on it, and started sending the rest of the recipes through my script. To my dismay, maybe 15% or so were usable, the rest were garbage. When I say usable, I do not mean good. I still had to do a lot of editing: fractions in the card would often be read as "%" or "yz" by OpenCV, and white space was pretty inconsistent. In the end, I picked 10-15 of my favorites, fixed the few that I had patience for, and copied the rest manually. Not a great experience, but I should only have to do it once, right? To demonstrate the end result, below is an example of an excellent-looking Oreo cheesecake recipe I found on Reddit:
* Recipes
** Oreo Cheesecake
:PROPERTIES:
:batch: 1
:course: dessert
:serves: 12
:END:
| Quantity | Unit | Ingredient | Notes |
|----------+------+----------------------------+---------------------------|
| 126 | g | All-Purpose Flour | |
| 31 | g | Unsweetened Cocoa Powder | |
| 375 | g | Sugar | |
| 1 | tsp | Baking Soda | |
| 0.5 | tsp | Baking Powder | |
| 1 | tsp | Salt | |
| 1 | tsp | Espresso Powder | |
| 3.5 | tsp | Vanilla Extract | |
| 120 | ml | Buttermilk | |
| 120 | ml | Coffee | Fresh brewed, strong, hot |
| 8 | | Oreos | |
| 114 | g | semi-sweet chocolate chips | |
| 385 | ml | Heavy Whipping Cream | |
| 24 | oz | Cream Cheese | Softened |
| 3 | | Eggs | |
| 240 | g | Sour Cream | |
| 14 | | Oreos | |
| 12 | | Oreos | Crushed |
| 100 | g | White Chocolate | |
| 70 | g | Powdered Sugar | |
| 12 | | Oreos | Chopped |
*** Instructions
1. Cake
- Preheat oven to 350°F (177°C).
- Grease one 9 inch cake pan, line with parchment paper, then grease the parchment paper.
- Whisk 110 g flour, cocoa powder, 175 g sugar, baking soda, baking powder, 0.5 tsp salt, and espresso powder together in a large bowl. Set aside.
- Using a handheld or stand mixer fitted with a whisk attachment or a hand whisk mix the oil, 1 egg, and 1 tsp vanilla together on medium-high speed until combined.
- Add the buttermilk and mix until combined.
- Pour the wet ingredients into the dry ingredients, add the hot coffee, and whisk or beat on low speed until the batter is completely combined and batter is thin.
- Place 8 Oreos into prepared pan, pour batter over Oreos (they will float more the faster you pour) and bake for 20-25 minutes or until a toothpick inserted in the center comes out mostly clean.
- Remove the cake from the oven and set on a wire rack. Allow to cool completely in the pan. Once completely cooled chill in the refrigerator for 6-24 hours.
2. Ganache
Instructions for Chocolate Ganache (this to be made immediately before mixing all your cheesecake ingredients)
- Heat 110 ml heavy whipping cream until it begins to boil, then pour it over the chocolate chips.
- Allow the mixture to sit for 3-4 minutes, then whisk until smooth.
3. Oreo Cheesecake
- Preheat oven to 325 degrees Fahrenheit.
- Boil a pot of water.
- Prepare a 9-inch leak-proof springform pan spraying it with non-stick baking spray and placing a parchment circle on the bottom.
- Grease the sides and top of the parchment paper.
- Use 3 or 4 pieces of foil around outside of springform to avoid water log during water bath.
- Place chocolate cake in prepared pan. Have a larger oven safe high walled pan for water bath handy. I use my 12 inch cast iron skillet. Set aside.
- In a large bowl, beat cream cheese for 1 minute.
- Add 200 g sugar and 16 g flour, mix until completely combined and there are no lumps. Scrape down the sides of the bowl with a spatula.
- With the mixer on low, add the eggs to the mixture, one at a time, and beat until just mixed through, careful not to over-beat. Scrape down the sides of the bowl with a spatula.
- Add 0.5 tsp salt, 0.5 tbsp vanilla extract and sour cream and beat until mixed through, stopping to scrape the sides and bottom of the bowl with a spatula.
- Stir in the crushed Oreo cookies.
- Spread slightly cooled ganache over cake that is in spring form pan you set aside
- Pour 1/3 of the batter into the prepared pan on top of cake.
- Arrange the 7 of the whole Oreo cookies on top of the cheesecake layer pour another 1/3 and arrange the other 7 then cover with the remaining batter. Level the top with a spatula. (If you have extra cheese cake batter you can bake in greased muffin tins or in another over safe container along side of your cheesecake. I use 2 metal bowls, the smallest for batter and the larger for water bath.)
- Place springform pan into water bath vessel then put into oven on center rack or bottom 1/3 of oven.
- Add the boiled water carefully so that it comes halfway up the outside of your springform. Bake for 60 minutes. The cheesecake should be just slightly wobbly in the center, but puffy and settled on the sides.
- Turn off the oven and crack open the door.
- Cool cheesecake completely in the open oven.
- Once completely cooled remove from the oven and refrigerate for at least 5-6 hours. Run a butter knife around the edges to release the cake from the pan.
- Store in the fridge until ready to add Oreo mousse layer.
4. Oreo Mousse
- Add the white chocolate chips and 45ml of heavy whipping cream to a small bowl.
- Microwave in 10 second increments until melted and smooth. Set aside to cool to about room temperature.
- Whip remaining 240ml of heavy whipping cream, powdered sugar and 1 tsp vanilla extract in a large mixer bowl fitted with the whisk attachment until stiff peaks form.
- Carefully fold about 1/3 of the whipped cream into the cooled white chocolate mixture until combined.
- Fold in the remaining whipped cream until well combined, then add the chopped Oreos.
- The springform outer ring will now need to be wiped clean and adjust so that the cake portion is exposed in order to fit and mold the mouse layer.
- Spoon the Oreo mousse on top of the cooled cheesecake and spread into an even layer.
- Place the cheesecake back in the fridge until the mousse layer is cold and firm, 3-4 hours.
- When the mousse layer is cold and firm, remove the cheesecake from the springform pan.
Each recipe has properties. The only one I'm not using right now is :course:, but it seemed like it could be helpful organizational metadata. The :batch: property represents how many batches of a recipe I usually make in one go. I almost always double a recipe to have leftovers, but rather than doubling everything in the ingredients list, I introduced this property to multiply each ingredient when I build my grocery list. The :serves: property (along with :batch:) tell me how much food I'm making.
Probably the most unusual design choice I made in this effort was using a table to enumerate ingredients. Ideally, I would use a list here like every other recipe I've ever seen, but this is not conducive to building a grocery list. I wanted to be able to combine ingredients if different recipes required the same thing, which meant parsing each item's quantity in the ingredient list. So rather than rely on regular expressions or white space to interpret the ingredient list and myself to enforce this implied structure rigorously, I thought the built-in structure of a table would be a lot easier.
Once I had all of my recipes, I started iterating on the export logic. After the second iteration, it occurred to me that I could do more than just produce a list of groceries, I could have each item organized by store. But I still spend a ton of time wandering around aisles in bigger stores, and it occurred to me that I can organize by aisle, too! Similar to recipe ingredients, I decided to put grocer information in a table, together with my export script in a Configuration heading:
* Configuration :noexport:
#+startup: fold
** Grocery List Generation Script...
** Grocery Store Ingredient Index
| Ingredient | Location | Aisle |
|------------+------------+-------|
| Bagels* | My Grocery | 4 |
The main thing to point out here, is that I decided to use a simplified regular expression to match ingredient names. This requires me being consistent and not make spelling mistakes, but that's typically not a problem for me. The downside to this is that this table needs to be filled out manually, but it has been a minor inconvenience so far. I just take notes in my grocery list on where things are and update the table when I get home. Admittedly, there was a lot of work to do with each trip to the store, but with each subsequent trip, updating the table takes exponentially less time. Truth be told, jotting down aisles makes grocery store trips kind of fun; I guess I like thinking about how next time I need this thing, I won't have to look for it.
So now I'll finally go over the actual subject of the post, the export script. I've tried to separate the code by task, and it's all enclosed in a let form to avoid polluting my Emacs configuration. Here's the context that all of the tasks are placed in:
(let* ((use-odds-and-ends t)
(grocer-list)
(ingredient-list)
(ingredient-tree)
(ingredient-string))
;; Code goes here
)
The first variable is a toggle named use-odds-and-ends which I set when I want to include things from a file I share with my spouse. This file is a checklist, and when we need something unrelated to recipes, we just check the item we want and add a quantity to the end. Here's an example:
* Food
- [ ] Arugula
- [X] Bagels 6
- [X] Bananas
The rest of the variables get populated by the script. The first thing the script does is parse the grocery store table into a list of plists and adds them to the grocer-list variable:
(org-element-map (org-element-parse-buffer 'element) 'table
(lambda (table)
(let* ((parent (org-element-property :parent table))
(gp (org-element-property :parent parent))
(heading (org-element-property :raw-value gp))
(begin (org-element-property :begin table))
(end (org-element-property :end table)))
(when (string= heading "Grocery Store Ingredient Index")
(dolist (row (nthcdr 2 (org-table-to-lisp (buffer-substring begin end))))
(let ((parsed `(:expr ,(nth 0 row) :store ,(nth 1 row) :aisle ,(string-to-number (nth 2 row)))))
(push parsed grocer-list)))))))
The plists themselves have a property for each column: :expr, :store, and :aisle.
Second, the script goes through the odds and ends if toggled, and adds them to the ingredient-list variable:
(when use-odds-and-ends
(let ((grocery-contents (with-temp-buffer
(insert-file-contents "~/org/share/groceries.org")
(org-element-parse-buffer)))
(grocery-string (with-temp-buffer
(insert-file-contents "~/org/share/groceries.org")
(buffer-string))))
(org-element-map grocery-contents 'plain-list
(lambda (list)
(dolist (properties-list (cdr list))
(when (string= (car properties-list) "item")
(let* ((properties (cadr properties-list))
(checkbox (plist-get properties :checkbox))
(contents-begin (plist-get properties :contents-begin))
(contents-end (plist-get properties :contents-end))
(has-store)
(item-string)
(quantity)
(ingredient))
(when (string= checkbox "on")
(setq item-string (string-trim (substring grocery-string (+ contents-begin -1) (+ contents-end -1))))
(let ((quantity-index (string-match " [0-9]+$" item-string)))
(if quantity-index
(progn
(setq quantity (string-to-number (substring item-string
quantity-index
nil)))
(setq item-string (substring item-string
0
quantity-index)))
(setq quantity 1)))
(setq ingredient `(:quantity ,quantity :name ,item-string))
(dolist (store grocer-list has-store)
(when (string-match (concat "^" (plist-get store :expr) "$")
(plist-get ingredient :name))
(plist-put ingredient :store (plist-get store :store))
(plist-put ingredient :aisle (plist-get store :aisle))
(setq has-store t)))
(when (not has-store)
(plist-put ingredient :store "Unsorted")
(plist-put ingredient :aisle 0))
(setf (alist-get item-string ingredient-list nil () 'string=) ingredient)
ingredient-list))))))))
This code goes through every checked list item in the file ~/org/share/groceries.org and adds it to the ingredient-list variable, which is a plist of plists. It checks if it knows which store to find the item at, and populates an ingredient plist whose properties are :quantity, :unit, :name and :notes. For grouping purposes, the key for each property pair in ingredient-list is its :name concatenated to its :unit. So the key for 1 teaspoon of sugar becomes sugartsp. The value is simply the ingredient plist. If no grocer is associated with the item, the :store property is set to Unsorted and the :aisle property is set to 0.
Third, the script goes through the recipes file (the file the script is in), and creates ingredient-list entries for marked recipes:
(org-element-map (org-element-parse-buffer 'element) 'table
(lambda (table)
(let* ((parent (org-element-property :parent table))
(gp (org-element-property :parent parent))
(state (org-element-property :todo-keyword gp))
(heading (org-element-property :raw-value gp))
(batch (org-element-property :BATCH gp))
(begin (org-element-property :begin table))
(end (org-element-property :end table)))
(when (string= state "TODO")
(dolist (row (nthcdr 2 (org-table-to-lisp (buffer-substring begin end))))
(let* ((ingredient `(:ingredient ,(nth 2 row) :quantity ,(string-to-number (car row)) :unit ,(nth 1 row)))
(ingredient-key (concat (plist-get ingredient :ingredient) (plist-get ingredient :unit)))
(logged-ingredient (alist-get ingredient-key ingredient-list nil () 'string=)))
(when (and batch (not (= (string-to-number batch) 1)))
(plist-put ingredient :quantity (* (plist-get ingredient :quantity) (string-to-number batch))))
(if logged-ingredient
(plist-put logged-ingredient :quantity (+ (plist-get logged-ingredient :quantity) (plist-get ingredient :quantity)))
(let ((has-store nil))
(dolist (store grocer-list has-store)
(when (string-match (concat "^" (plist-get store :expr) "$")
(plist-get ingredient :ingredient))
(plist-put ingredient :store (plist-get store :store))
(plist-put ingredient :aisle (plist-get store :aisle))
(setq has-store t)))
(when (not has-store)
(plist-put ingredient :store "Unsorted")
(plist-put ingredient :aisle 0))
(setf (alist-get ingredient-key ingredient-list nil () 'string=) ingredient)))))))))
Any recipe heading marked with TODO has its ingredients table parsed. Each ingredient's :quantity is multiplied by the :batch: property on the Org heading, and the ingredient is added to ingredient-list following the same key-value convention for odds and ends. Its :store and :aisle properties are set the same way as odds and ends as well. The one caveat is that if the key for an ingredient is already present, the :quantity of the current ingredient is just added to the :quantity of the existing ingredient. Since the unit is included in the key, adding teaspoons to tablespoons (or whatever) isn't a concern!
Fourth, the ingredient list is sorted by its keys so ingredients are organized by their unit:
(setq ingredient-list (sort ingredient-list (lambda (first second) (string< (car first) (car second)))))
Fifth, the script transforms the ingredient-list into an ingredient-tree: Once the ingredients are sorted, they have to be put into a hierarchical list somehow; the easiest way I could think of to do this was to mapconcat nested lists (specifically alists):
(dolist (ingredient-kv ingredient-list t)
(let* ((ingredient (cdr ingredient-kv))
(stored-ingredient (alist-get (plist-get ingredient :aisle) (alist-get (plist-get ingredient :store) ingredient-tree nil () 'string=))))
(setf (alist-get (plist-get ingredient :aisle) (alist-get (plist-get ingredient :store) ingredient-tree nil () 'string=)) (append stored-ingredient `(,ingredient)))))
Sixth, the script generates a string containing the Org checklist with all the ingredients:
(setq ingredient-string (format "%s"
(mapconcat (lambda (store-tree)
(concat "* "
(car store-tree)
"\n\n"
(mapconcat (lambda (aisle-tree)
(let ((aisle (car aisle-tree)))
(concat (when (> aisle 0)
(concat "Aisle " (number-to-string aisle) "\n\n"))
(mapconcat (lambda (ingredient)
(concat "- [ ] "
(replace-regexp-in-string "\\.0+$" "" (number-to-string (plist-get ingredient :quantity)))
(when (not (string= (plist-get ingredient :unit) ""))
(concat " "
(plist-get ingredient :unit)))
" "
(plist-get ingredient :name)))
(cdr aisle-tree)
"\n"))))
(sort (cdr store-tree) (lambda (a b) (< (car a) (car b))))
"\n\n")))
ingredient-tree
"\n\n")))
I apologize for all the nesting; I guess that's what I get for nesting alists. I wish this block were more readable, but I want to avoid defining functions in Emacs that aren't in my core config. Justifications aside, I don't have a lot to say about this block, but here's what the output looks like:
* Unsorted
- [ ] 4 Bell Pepper
- [ ] 16 oz Broccoli Florets
- [ ] 1 oz Cashews
* My Grocer
- [ ] 16 oz Arugula
- [ ] 6 Bananas
** Aisle 5
- [ ] 1 loaf Bread
If no aisle is associated with an ingredient for some reason, it just sits at the top of the store heading.
The file exports to a share folder on my WebDAV server. I pick it up on my phone and view it with Orgzly. When it gets used it works great! I don't mind going to the grocery store anymore, and sometimes I even enjoy it. I have some other ideas on how to use Org to streamline my day-to-day; I'll be happy if they're half as successful as this has been.