Working alone is so exhausting so I created my own assistant
#javascript#node#codenewbie#showdev
Min

Min @choisohan

About: Artist, Creative Coder

Location:
Vancouver, BC
Joined:
Mar 18, 2020

Working alone is so exhausting so I created my own assistant

Publish Date: Mar 15 '22
756 58

Working alone is great... but... I am so tired of dealing with all of these.. these shitty uninspiring, repetitive, tedious tasks...

I didn't realize how many emails I had to reply to and how many invoices I had to make on my own. How many tasks do I have to manage and schedule?!

I sometimes freaked out alone because I was too busy, and sometimes I freaked out because suddenly there's nothing to do..

Console.log( my_life )
Enter fullscreen mode Exit fullscreen mode

I need something stabilizes me all the time no matter what happens to me. Maybe I need a manager. Manage me! (Funny to say that because I left work to not be managed by others.LOL)

For a sec, I think about hiring somebody, but I don't have money for that.(of course)

So here is my attempt to create my personal assistant with discord and notion API and, what I learned from this project.

If there is anything incorrect, please let me know. I'd love to learn from you!

1. I learned NLP

The first thing in my head to create a chatbot is how a chatbot analyze the intention of my text. People say the same thing in various forms of expression

  • How are you?
  • How's it going?
  • What's up?

I can write the code like if( text.includes('how are you') ) for every expression, but how inefficient would it be? This is the time when NLP is useful.

Natural language processing (NLP) is the ability of a computer program to understand human language as it is spoken and written -- referred to as natural language. It is a component of artificial intelligence (AI). NLP has existed for more than 50 years and has roots in the field of linguistics. from techtarget.com

While I still don't fully understand how NLP works, my understanding is so far like this.

  1. A developer submits a set of sentences future users might send

  2. There are three types of variables A user can assign for each text.
    (1) intension: The most important part of the sentence. Only one can be assigned for one sentence. ex) I need a new iPhone! -> Request_add_wishlist
    (2) Entities: This is a smaller assembly piece, basically a critical keyword to define the intention. ex) I need shopping! -> need, shopping.
    (3) Traits : ( I'll ignore this part for now )

  3. An NLP program analyzes what kind of entity composition is for specific intension.

  4. Next time a user send complete new text, a trained NLP will spit out what a user means.

Among multiple different services to provide NLP, I chose to use wit.ai service, which is an open-source service developed by Facebook, Meta.

The useful thing about wit.ai is chatting text I sent by discord also showed up on the wit.ai training session, so even if a bot failed to understand what I meant, I can always go to wit and correct it.

The downside is documentation is not so clear, so... anyway... it's alright. I will take it...

2. I learned Discord.js

In the end, I chose to create a discord bot over a Slack bot even tho I am not really a discord user.

Some servers I joined to get help usually has too many channels on the sidebar, and too many discussion is going on. It just intimates too much. Like you entered in a club and see full of people dancing a Michael Jackson thriller perfectly, but I don't know that damn dance. "Quick, get out from that club!"

Anyway, during this project, I could learn about discord.js and at the same time a little bit more about discord itself...

npm install discord.js
Enter fullscreen mode Exit fullscreen mode

The exciting thing about discord js is it gives so many options to interact with other users. Can create buttons, slash commands, interact with emoji etc.. way more than just sending a message.

The most useful message type is Embed Message.

( I set my coordinator as Tahani from the TV series 'The good place' because I love her character on the show so much. so jolly!)

Embed message returns the beautifully formatted information. You can directly add the field like this.

_embed.addFields({name : "Count", value : Count })
Enter fullscreen mode Exit fullscreen mode

or you can create the dictionary and loop for each keys

Object.keys(style).forEach( k=>{
    _newEmbed.addFields({name : k , value :style[k].toString() })
})
Enter fullscreen mode Exit fullscreen mode

The useful reference for discord bot -> Discord JS Guide

3. I learned Notion API

I love notionHQ. Actually, I am obsessed with it. I've been using it since it was not the mainstream. Recently I heard it acquired automate.io. Alight, NotionHQ, Let's go! Bigger and bigger! But there were a few moments.. when I was considering leaving Notion... it's because Notion doesn't have a recurring task option!

Like I wrote in the intro, repeating and repeating and repeating tasks is already so boring, and I have to create the checkbox of it every time? Oh shit, I don't want that.

I left the Notion and came back again over and over, like a troubled teenager arguing with a parent. A year ago, Notion released notion API; "This is the final, guys. I am going to build an automation system for you and going to be with you happily ever after 4everr."

npm install @notionhq/client

Notion API is great! but unfortunately, there are a few limits for (v1.0.3) right now.

  1. Editing a block is not available yet. Basically, if you want to tick the checkbox block, you should delete the current checkbox item and append a duplicated block again. It's not impossible but kinda annoying. Duplicating a single block is not such a bad case, but how about if you want to duplicate a whole page? Sadly you still need to delete a whole page and duplicate it. If the page has a children element, the problem gets more annoying.

  2. Get children to block only return the top-level items on the hierarchy.

   var children =  await NOTION.blocks.children.list({ block_id: _block.id });
Enter fullscreen mode Exit fullscreen mode

so for example, if there's a notion block like this

The code output will be only [block_A, block_B]. So even if you duplicate the parent with its children, [block_Aa, block_Ab, block_Ba,block_Bb] won't be duplicated.

If you want all the children under parent, there's no other way than keep asking children if they has grand-grand children…

​

   if( block_A.hasChildren ){
    children.push( await NOTION.blocks.children.list({ block_id: block_A.id }) )
   }
Enter fullscreen mode Exit fullscreen mode
  1. Some block type is not supported. Just think non-native notion blocks such as embed tweet or embed Figma are not supported, which is totally fine, but. butttt. code blocks are also not supported, so I felt kinda sad. :(

All i want is block to save the code so I can run it by ‘eval()’ which can be just any text block. Just not pretty as I wished, that’s all. Haha

var scripts = blocks.filter( block => Object.keys(block)[0] == "callout"  )
await eval(scripts[0])
Enter fullscreen mode Exit fullscreen mode

Every lunch time, recipe inspiration function runs.

4. I learned about Cron

One of the must-have features for the coordinator bot was sending scheduled messages, so I can do the task on time.

To do so, I got to know Cron.

npm i cron
Enter fullscreen mode Exit fullscreen mode

A CRON expression is a string comprising five or six fields separated by white space[16] that represents a set of times, normally as a schedule to execute some routine.
from wikipedia

***??... Another regular expression all over again.. horror.. But actually, the breakdown of cron time is pretty simple, min + hour + date + month + week.

Now I should generate cron time by ‘Date()’. This time, instead of writing in js, I wrote as notion built-in formula so even if I update the schedule by notion, the clon time value also can be updated at the same time.

min : 
empty(prop("Unit")) ? if(minute(prop("Date")) + minute(prop("Date")) == 0, format(minute(prop("Edited"))), format(minute(prop("Date")))) : if(prop("Unit") == "minute", "*/" + format(if(empty(prop("Recurring")), 1, prop("Recurring"))), if(prop("Unit") != "hour" or prop("Unit") != "minute", if(empty(prop("Date")), if(empty(prop("Date")), format(minute(prop("Edited"))), format(minute(prop("Date")))), format(minute(prop("Date")))), "*"))
Enter fullscreen mode Exit fullscreen mode

Such a ugly code, for now but so far it seems working as wish. (full code is on my repo)

While I can create a new scheduled event by Notion, I wished to create the new reminder by chat as well. Luckily Wit.ai has built-in entities called "duration" and "datetime", such a lifesaver!

Text to clean format

5. I learned Puppeteer

I started to feel more greedy and needy. Hey, bot, you can give weather and different time zone time, my today's tasks and project due date now.. how about.. recommendation for today's meal?? M.M.M

First, I found the biggest recipe API called Spoonacular API, but .. the food pictures look kinda.. not really inspiring..

So instead of food API, I decided to fetch the information directly from my favourite cooking site. I heard the Puppeteer is good for web scraping, I always wanted to try, but this is my first time trying Puppeteer!

npm i puppeteer
Enter fullscreen mode Exit fullscreen mode

Puppeteer is such an amazing package, but there were a few confusing parts though.

When I tried to get element by document.querySelector() .this returned undefined or null object. I was panicked, and I thought Puppeteer failed to find the element, but it's there.. when I specifically ask textContent or src, href?

await page.evaluate(()=>{
    return document.querySelectorAll(".selector").map( sel => sel.textContent)
});
Enter fullscreen mode Exit fullscreen mode

As I understand is what Puppeteer gives me is not an HTML element but something else.. ( is there anybody explain a bit easy for me? i am helpless lol)

Anyway, now I get a recommendation for a recipe. yay

6. I learned Heroku

Because my bot will be deactivated when I don't run bot.js, I needed Heroku to serve it on the cloud.

While I am hosting my sites on Netlify, I am still not familiar with the hosting service. I misunderstood Heroku and Netlify as the same kind of service, but this time, I learned they are not. Like this whole post is such lack of professional knowledge, this is my short understanding about Netlify and Heroku so far

  • Netlify is more for the front-end, and Heroku is more for the back-end
  • Netlify is always active, but Heroku fall asleep...zzz
  • Netlify is site hosting online, but Heroku is run the script on Linux
  • Netlify start charging by build time, Heroku charging with something called Dyno hours

Dyno is a container that runs the command in Linux. The busier the app will be, the more dyno is required to be purchased. Because I am a free user, I have a single dyno.

Free user has 1000 free dyno hours. It means that even if my app runs consistently without resting for the whole month, it wouldn't get over 730 hours. Free is always good. It makes me feel cosy.😊

Concluding

My Git Repo

While I want to improve more and add more features, my Tahani bot works quite wonderfully right now. Great to have a companion! (sad to say that. lol)

I tried a lot of new things this time, I learned so much; however, I also feel like I didn't spend enough time to learn about each of them more precisely. Maybe that's what I'm going to do with next dev.to

Besides having a customised virtual assistant for my daily routine, the knowledge I earned was so valuable. There are so much potential to improve my interactive character project
Thank you for reading this messy post. I am not a trained developer (I a visual artist), but I am very excited to share thoughts and get to know more about creating something cool.

Please comment on anything if you want to correct my post and educate me. It would be awesome to learn from all of you!

Comments 58 total

  • Marissa B
    Marissa BMar 15, 2022

    Good job on learning so much new stuff! That was a lot of steps and your story from start to finish was clear. I might have to make one of these.

    • Min
      MinMar 15, 2022

      Really fun project, you should try and let me know your progress!

  • SVGator
    SVGatorMar 15, 2022

    Impressive, to say the least!

    • Min
      MinMar 15, 2022

      Thank you!

  • lionel-rowe
    lionel-roweMar 15, 2022

    As I understand is what Puppeteer gives me is not an HTML element but something else.. ( is there anybody explain a bit easy for me? i am helpless lol)

    The function passed as a callback runs in the context of the headless browser, with all the relevant web APIs available. So within that function, document.querySelector gives a real live DOM element that you can manipulate however you like:

    const isItARealDiv = await page.evaluate(() => {
    return document.querySelector("div") instanceof HTMLDivElement
})

isItARealDiv // true
    Enter fullscreen mode Exit fullscreen mode

    The problem is that when passing the return value back to the parent script, everything is serialized and then deserialized again — something similar to JSON.parse(JSON.serialize(result)). DOM elements can't be properly serialized, getting converted to undefined, so you need to return only the serializable data that you need (text content, specific attributes, inner/outer HTML, etc).

    • Min
      MinMar 15, 2022

      Thank you Lionel-rowe! ( Still confused lol)

      • lionel-rowe
        lionel-roweMar 15, 2022

        The parent puppeteer script runs in the Node.JS runtime, whereas the callback to page.evaluate runs in the Chromium browser runtime, headlessly by default ("headless" basically just means that it runs in the background, so you can't visibly see it running). Passing complex data between runtimes is often not possible, because the different runtimes don't know how to interpret it, and DOM elements (DOM is the way the browser interprets HTML) are internally very complex. So to simplify the message passing, Puppeteer uses a serialized format that both runtimes can easily understand. The drawback is that any data that can't be converted to this serialized format is lost.

        You can think of "serialized" as meaning something like flat, like a string of letters or binary digits. JSON is a typical serialization format and is useful because it allows the "flattening" of "deep" structures. For example, the JavaScript object { a: { b: 1 } } nests b within a, yet it can be serialized to the JSON string {"a":{"b":1}}. Why is this flat? Well, it's simply the character {, followed by ", followed by a, etc., so it can be read left-to-right; even though the object it represents is a tree structure.

        Puppeteer does much of this JSON serialization "under the hood", so you often don't need to worry about it; but JSON can't serialize DOM nodes, because they contain circular structures, e.g. *{ a: { b: *{ a: ... } } } (where * represents a reference to the exact same object). So you need to return only things that JSON can represent — strings, numbers, booleans, null, arrays, and objects containing other JSON-able stuff.

        const elementData = await page.evaluate(() => {
    const el = document.querySelector('h1')

    return {
        textContent: el.textContent, // string — OK
        childElementCount: el.childElementCount, // number — OK
        className: el.className, // string — OK
        outerHTML: el.outerHTML, // string — OK
    }
})

console.log(elementData)
// {
//     textContent: 'Posted on Mar 15'
//     childElementCount: 1,
//     className: 'fs-xs color-base-60'
//     outerHTML: '<p class="fs-xs color-base-60">Posted on <time datetime="2022-03-15T02:18:47Z" class="date-no-year" title="Tuesday, March 15, 2022, 2:18:47 AM">Mar 15</time></p>',
// }
        Enter fullscreen mode Exit fullscreen mode
        • Min
          MinMar 18, 2022

          Omg @lionelrowe
          Big thank you to explain this with full of kind detail!
          This is really easy to understand! You are the best!👍👍👍👍👍

      • Cody Swartz
        Cody SwartzMar 17, 2022

        I think Puppeteer and Selenium both suffer from the same problem -- Selenium has been around longer, likely before websites were more complicated with single page magic and dynamic content. The last thing I remembered is that if you query for something before it exists/mounts/renders you'll get nothing back, so you need to wait/poll for it to be available. I thought Puppeteer has helpers around this, or this is why people start to reach for additional libraries on top of these tools to help with this problem. It's been a while since I've touched Puppeteer or Selenium, but I do remember the pains of working with them in single page applications.

        Edit: puppeteer.github.io/puppeteer/docs...

        • lionel-rowe
          lionel-roweMar 17, 2022

          @ctsstc yeah, Puppeteer gives you various APIs, such as page.waitForSelector, to deal with that, but it can be finnicky knowing exactly what you need to wait for and avoiding race conditons.

  • Ashwin V
    Ashwin VMar 15, 2022

    Nice, I learned many terms from this post! Keep going!😊

    • Min
      MinMar 15, 2022

      Thank you! 🥰

  • Fumagalli
    FumagalliMar 15, 2022

    This project is incredible!

    • Min
      MinMar 15, 2022

      Thank you for saying that!🤗

  • JoelBonetR 🥇
    JoelBonetR 🥇Mar 15, 2022

    Love it 😂 i did a discord weekend project in the past and just for fun and this seems more like a month project.
    Did you published it? Can i get a link to include it to my discord server? 🤩

    • Min
      MinMar 15, 2022

      It's not published because I didn't set the notion authentification per user.
      it would be so embarrasing if people manage their tasks on my notion. lol

      • JoelBonetR 🥇
        JoelBonetR 🥇Mar 16, 2022

        Hahaha and what about the fun? 😅

        • Min
          MinMar 16, 2022

          I will consider to publish it :)

  • Marco Moscatelli
    Marco MoscatelliMar 15, 2022

    What node js library did you use for NPL?

  • Dhravya
    DhravyaMar 15, 2022

    Damn, this is an amazing project! Great work. Amazing discord bot!

    • Min
      MinMar 15, 2022

      Thank you for saying that🥰

  • nebocoder
    nebocoderMar 15, 2022

    Thank you for this wonderful post. I planned on learning Discord.js to build a personal bot eventually, and I'm so glad I discovered this!

    • Min
      MinMar 15, 2022

      Please show what you make 😀

  • Dan Walsh
    Dan WalshMar 15, 2022

    Super impressive feat! It’s one thing to build something new; it’s a whole other thing to learn several libraries/systems in the process. Great work, never stop learning! 😇

    • Min
      MinMar 15, 2022

      hehe thank you very much Dan!

  • tq-bit
    tq-bitMar 16, 2022

    I love the idea. If you're into shares, you might as well give the Yahoon finance API a shot. It gives you 100 free requests per day, that's more than enough for a single person. For instance, I've built a telegram bot earlier that tells me when is a good time to buy & sell

    yahoofinanceapi.com/

    • Min
      MinMar 16, 2022

      Ah-ha! that’s nice idea. Maybe i can add that!

  • 🚩 Atul Prajapati 🇮🇳
    🚩 Atul Prajapati 🇮🇳Mar 16, 2022

    Wow my you are the Tony stark

    • Min
      MinMar 16, 2022

      Actually I considered to name it as Jarvis 😂

    • Timmortal
      TimmortalMar 25, 2022

      To make a "Jarvis" that follows you from conception to birth and before you merge back in to the universe, is my dream

  • Akin C.
    Akin C.Mar 16, 2022

    Hello Min,

    thank you for your article.
    I've never looked into implementing chatbots, so this is interesting for me.
    I like your creative but simple idea of ​​having a digital companion. I don't think that's sad, it's wise, because like you said
    "I am so tired of dealing with all of these.. these shitty uninspiring, repetitive, tedious tasks...".

    • Min
      MinMar 18, 2022

      hehe thank you to saying that! Akin!
      very glad to hear i inspire somebody with my idea 🥰

  • Priyab Dash
    Priyab DashMar 16, 2022

    Hell I love this idea, now I need to implement it in Python and slack

    • Min
      MinMar 18, 2022

      Thank you very much! I want to see your version! please share the progress! 😊

  • Tom Smykowski
    Tom SmykowskiMar 17, 2022

    Congrats on your project. It is extremely epic. Now please monetize it :-)

    • Min
      MinMar 18, 2022

      Thanks for saying it's epic haha

  • Andras Mihaly
    Andras MihalyMar 17, 2022

    Wow, I really enjoyed reading through your article! Great work on the project, this is pretty impressive! Thanks for sharing @happping_min !

    • Min
      MinMar 18, 2022

      Thank you very much Andras!😊

  • Bill Raymond
    Bill RaymondMar 17, 2022

    This was an inspiring and well written article. Thank you!

    • Min
      MinMar 17, 2022

      Thank you very much Bill!

  • Min
    MinMar 18, 2022

    haha I tried siri and google assitant. They sucks because there no way to customize.
    I am glad to hear you found this is useful. show your progress when you make any progress. I am sure you can make it better one!

  • Min
    MinMar 18, 2022

    Thanks Leonid! Yeah I like what I am doing haha

  • Jackson Kasi
    Jackson KasiMar 18, 2022

    wow, that's cool...

    • Min
      MinMar 18, 2022

      Thank you for saying that🥰

  • Anneta Wamono
    Anneta WamonoMar 18, 2022

    This is a great project! And I've been meaning to make a discord bot and you've really broken the process down quite well.

    • Min
      MinMar 18, 2022

      Haha thanks! Great to hear my article helps others!

  • Moses
    MosesMar 19, 2022

    1- great work, you built it using different languages than we did.
    2-i see photo of that indian hot chick. dream about it to be your assitant. :)
    3- we did it differently, ajax, js, prolog, aiml used in our bot. she has intelligence
    check it out in our website. lots of videos there.
    4- send me a link to go and talk with her to see how she is doing.
    email: support at robot-hosting.com
    have a happy time
    Moses

  • Brian Kirkpatrick
    Brian KirkpatrickMar 20, 2022

    This is great stuff! I love the way you approached an under-constrained problem and found a value-added approach that contributed real help to your workflow.

    One thing I found when going through your steps was, of course, that different developers might want different things from an automated assistant. How did you decide what you wanted an "assistant" to do for you--that is, what were your objectives? Are there specific tedious things you seek to automate for your own workflow versus collaborative tools that support team-based efforts?

    Two thumbs up!

  • darmaji.code.v
    darmaji.code.vMar 22, 2022

    Thank You

  • Prodsllc
    ProdsllcMar 22, 2022

    Thank you as well Lionel-rowe.

  • Andrew Baisden
    Andrew BaisdenMay 12, 2022

    What a cool concept!

  • Cindy Bahl
    Cindy BahlDec 20, 2022

    Wow. Impressive. I was going to ask if you tried Character.AI for this but 'duh' it isn't meant for this type of thing. But, wow, this is cool!

Add comment