Filtered by JavaScript

Page 5

Reset

How I upload Firebase images optimized

September 2, 2021
0 comments JavaScript, Web development, Firebase

I have an app that allows you to upload images. The images are stored using Firebase Storage. Then, once uploaded I have a Firebase Cloud Function that can turn that into a thumbnail. The problem with this is that it takes a long time to wake up the cloud function, the first time, and generating that thumbnail. Not to mention the download of the thumbnail payload for the client. It's not unrealistic that the whole thumbnail generation plus download can take multiple (single digit) seconds. But you don't want to have the user sit and wait that long. My solution is to display the uploaded file in a <img> tag using URL.createObjectURL().

The following code is most pseudo-code but should look familiar if you're used to how Firebase and React/Preact works. Here's the FileUpload component:


interface Props {
  onUploaded: ({ file, filePath }: { file: File; filePath: string }) => void;
  onSaved?: () => void;
}

function FileUpload({
  onSaved,
  onUploaded,
}: Props) => {
  const [file, setFile] = useState<File | null>(null);

  // ...some other state stuff omitted for example.

  useEffect(() => {
    if (file) {
      const metadata = {
        contentType: file.type,
    };

    const filePath = getImageFullPath(prefix, item ? item.id : list.id, file);
    const storageRef = storage.ref();

    uploadTask = storageRef.child(filePath).put(file, metadata);
    uploadTask.on(
      "state_changed",
      (snapshot) => {
        // ...set progress percentage
      },
      (error) => {
        setUploadError(error);
      },
      () => {
        onUploaded({ file, filePath });  // THE IMPORTANT BIT!

        db.collection("pictures")
          .add({ filePath })
          .then(() => { onSaved() })

      }
    }
  }, [file])

  return (
      <input
        type="file"
        accept="image/jpeg, image/png"
        onInput={(event) => {
          if (event.target.files) {
            const file = event.target.files[0];
            validateFile(file);
            setFile(file);
          }
        }}
      />
  );
}

The important "trick" is that we call back after the storage is complete by sending the filePath and the file back to whatever component triggered this component. Now, you can know, in the parent component, that there's going to soon be an image reference with a file path (filePath) that refers to that File object.

Here's a rough version of how I use this <FileUpload> component:


function Images() {

  const [uploadedFiles, setUploadedFiles] = useState<Map<string, File>>(
    new Map()
  );

  return (<div>  
    <FileUpload
      onUploaded={({ file, filePath }: { file: File; filePath: string }) => {
        const newMap: Map<string, File> = new Map(uploadedFiles);
        newMap.set(filePath, file);
        setUploadedFiles(newMap);
      }}
      />

    <ListUploadedPictures uploadedFiles={uploadedFiles}/>
    </div>
  );
}

function ListUploadedPictures({ uploadedFiles}: {uploadedFiles: Map<string, File>}) {

  // Imagine some Firebase Firestore subscriber here
  // that watches for uploaded pictures. 
  return <div>
    {pictures.map(picture => (
      <Picture picture={picture} uploadedFiles={uploadedFiles} />
    ))}
  </div>
}

function Picture({ 
  uploadedFiles,
  picture,
}: {
  uploadedFiles: Map<string, File>;
  picture: {
    filePath: string;
  }
}) {
  const thumbnailURL = getThumbnailURL(filePath, 500);
  const [loaded, setLoaded] = useState(false);

  useEffect(() => {
    const preloadImg = new Image();
    preloadImg.src = thumbnailURL;

    const callback = () => {
      if (mounted) {
        setLoaded(true);
      }
    };
    if (preloadImg.decode) {
      preloadImg.decode().then(callback, callback);
    } else {
      preloadImg.onload = callback;
    }

    return () => {
      mounted = false;
    };
  }, [thumbnailURL]);

  return <img
    style={{
      width: 500,
      height: 500,
      "object-fit": "cover",
    }}
    src={
      loaded
        ? thumbnailURL
        : file
        ? URL.createObjectURL(file)
        : PLACEHOLDER_IMAGE
    }
  />
}

Phew! That was a lot of code. Sorry about that. But still, this is just a summary of the real application code.

The point is that; I send the File object back to the parent component immediately after having uploaded it to Firebase Cloud Storage. Then, having access to that as a File object, I can use that as the thumbnail while I wait for the real thumbnail to come in. Now, it doesn't matter that it takes 1-2 seconds to wake up the cloud function and 1-2 seconds to perform the thumbnail creation, and then 0.1-2 seconds to download the thumbnail. All the while this is happening you're looking at the File object that was uploaded. Visually, the user doesn't even notice the difference. If you refresh the page, that temporary in-memory uploadedFiles (Map instance) is empty so you're now relying on the loading of the thumbnail which should hopefully, at this point, be stored in the browser's native HTTP cache.

The other important part of the trick is that we're using const preloadImg = new Image() for loading the thumbnail. And by relying on preloadImage.decode ? preloadImage.decode().then(...) : preload.onload = ... we can be informed only when the thumbnail has been successfully created and successfully downloaded to make the swap.

How to fadeIn and fadeOut like jQuery but with Cash

August 24, 2021
0 comments JavaScript

Remember jQuery? Yeah, it was great. But it was also horrible in its own ways but only when compared to the more powerful tools that we have now as of 2021. I still (almost) use it here on my site. Atually, I use "fork" of jQuery called Cash which calls itself: "An absurdly small jQuery alternative for modern browsers."
Cash is written in TypeScript, which gives me peace of mind, and as a JS bundle, it's only 19KB minified (5.3KB Brotli compressed) whereas jQuery is 87KB minified (27KB Brotli compressed).

But something that jQuery has, that Cash doesn't, is animations. E.g. $('myselector').fadeIn(). If you need to do this with Cash you can use the following pure JavaScript solution:


// Example implementation

const msg = $('<div class="message">')
  .text(`Random message: ${Math.random()}`)
  .css("opacity", 0)
  .css("transition", "opacity 600ms")
  .prependTo($("#root"));
setTimeout(() => msg.css("opacity", 1), 0);

setTimeout(() => {
  msg.css("transition", "opacity 1000ms").css("opacity", 0);
  setTimeout(() => msg.remove(), 1000);
}, 3000);

What this application demonstrates is the creation of a <div> that's immediately injected into the DOM but slowly fades into view. And 3 seconds later it fades out and is removed. Full demo/sample application here.

Sample application using cash like jQuery's $.fadeIn().

The point of the demo is how you can cause the fade-in effect with just Cash but still relies on CSS for the actual animation.
The trick is to, ultimately, create it first like this:


<div class="message" style="opacity:0; transition: opacity 600ms">
  Random message: 0.6517198324628395
</div>

and then, right after it's been added to the DOM, change the style=... to:


-<div class="message" style="opacity:0; transition: opacity 600ms">
+<div class="message" style="opacity:1; transition: opacity 600ms">

What's neat about this is that you use the transition shortcut so it's done entirely with CSS instead of a requestAnimationFrame and/or while-loop like jQuery's effects.js does it.

Note! This is not a polyfill since jQuery's fadeIn() (etc.) can do a lot more such as callbacks. The example might not be great but I hope this little solution becomes useful for someone else who needs this.

How to submit a form with Playwright

August 3, 2021
0 comments JavaScript

Because it was driving me insane, and because I don't want to ever forget...

Playwright is a wonderful alternative to jest-puppeteer for doing automated headless browser end-to-end testing. But one I couldn't find in the documentation, Google search, or Stackoverflow was: How do you submit a form without clicking a button?. I.e. you have focus in an input field and hit Enter. Here's how you do it:


await page.$eval('form[role="search"]', (form) => form.submit());

The first part is any CSS selector that gets you to the <form> element. In this case, imagine it was:


<form action="/search" role="search">
  <input type="search" name="q">
</form>

You, or my future self, might be laughing at me for missing something obvious but this one took me forever to solve so I thought I'd better blog about it in case someone else gets into the same jam.

UPDATE (Sep 2021)

I found a much easier way:


await page.keyboard.press("Enter");

This obviously only works when you've typed something into an input so the focus is on that <input> element. E.g.:


await page.fill('input[aria-label="New shopping list item"]', "Carrots");
await page.keyboard.press("Enter");

What English stop words overlap with JavaScript reserved keywords?

May 7, 2021
2 comments JavaScript, MDN

The list of stop words in Elasticsearch is:

a, an, and, are, as, at, be, but, by, for, if, in, into, 
is, it, no, not, of, on, or, such, that, the, their, 
then, there, these, they, this, to, was, will, with

The list of JavaScript reserved keywords is:

abstract, arguments, await, boolean, break, byte, case, 
catch, char, class, const, continue, debugger, default, 
delete, do, double, else, enum, eval, export, extends, 
false, final, finally, float, for, function, goto, if, 
implements, import, in, instanceof, int, interface, let, 
long, native, new, null, package, private, protected, 
public, return, short, static, super, switch, synchronized, 
this, throw, throws, transient, true, try, typeof, var, 
void, volatile, while, with, yield

That means that the overlap is:

for, if, in, this, with

And the remainder of the English stop words is:

a, an, and, are, as, at, be, but, by, into, is, it, no, 
not, of, on, or, such, that, the, their, then, there, 
these, they, to, was, will

Why does this matter? It matters when you're writing a search engine on English text that is about JavaScript. Such as, MDN Web Docs. At the time of writing, you can search for this because there's a special case explicitly for that word. But you can't search for for which is unfortunate.

But there's more! I think we should consider certain prototype words to be considered "reserved" because they are important JavaScript words that should not be treated as stop words. For example...

How to simulate slow lazy chunk-loading in React

March 25, 2021
0 comments React, JavaScript

Suppose you have one of those React apps that lazy-load some chunk. It just basically means it injects a .js static asset URL into the DOM and once it's downloaded by the browser, it carries on the React rendering with the new code loaded. Well, what if the network is really slow? In local development, it can be hard to simulate this. You can mess with the browser's Devtools to try to slow down the network, but even that can be too fast sometimes.

What I often do is, I take this:


const SettingsApp = React.lazy(() => import("./app"));

...and change it to this:


const SettingsApp = React.lazy(() =>
  import("./app").then((module) => {
    return new Promise((resolve) => {
      setTimeout(() => {
        resolve(module as any);
      }, 10000);
    });
  })
);

Now, it won't load that JS chunk until 10 seconds later. Only temporarily, in local development.

I know it's admittedly just a hack but it's nifty. Just don't forget to undo it when you're done simulating your snail-speed web app.

PS. That resolve(module as any); is for TypeScript. You can just change that to resolve(module); if it's regular JavaScript.

In JavaScript (Node) which is fastest, generator function or a big array function?

March 5, 2021
0 comments Node, JavaScript

Sorry about the weird title of this blog post. Not sure what else to call it.

I have a function that recursively traverses the file system. You can iterate over this function to do something with each found file on disk. Silly example:


for (const filePath of walker("/lots/of/files/here")) {
  count += filePath.length;
}

The implementation looks like this:


function* walker(root) {
  const files = fs.readdirSync(root);
  for (const name of files) {
    const filepath = path.join(root, name);
    const isDirectory = fs.statSync(filepath).isDirectory();
    if (isDirectory) {
      yield* walker(filepath);
    } else {
      yield filepath;
    }
  }
}

But I wondered; is it faster to not use a generator function since there might an overhead in swapping from the generator to whatever callback does something with each yielded thing. A pure big-array function looks like this:


function walker(root) {
  const files = fs.readdirSync(root);
  const all = [];
  for (const name of files) {
    const filepath = path.join(root, name);
    const isDirectory = fs.statSync(filepath).isDirectory();
    if (isDirectory) {
      all.push(...walker(filepath));
    } else {
      all.push(filepath);
    }
  }
  return all;
}

It gets the same result/outcome.

It's hard to measure this but I pointed it to some large directory with many files and did something silly with each one just to make sure it does something:


const label = "generator";
console.time(label);
let count = 0;
for (const filePath of walker(SEARCH_ROOT)) {
  count += filePath.length;
}
console.timeEnd(label);
const heapBytes = process.memoryUsage().heapUsed;
console.log(`HEAP: ${(heapBytes / 1024.0).toFixed(1)}KB`);

I ran it a bunch of times. After a while, the numbers settle and you get:

  • Generator function: (median time) 1.74s
  • Big array function: (median time) 1.73s

In other words, no speed difference.

Obviously building up a massive array in memory will increase the heap memory usage. Taking a snapshot at the end of the run and printing it each time, you can see that...

  • Generator function: (median heap memory) 4.9MB
  • Big array function: (median heap memory) 13.9MB

Conclusion

The potential swap overhead for a Node generator function is absolutely minuscule. At least in contexts similar to mine.

It's not unexpected that the generator function bounds less heap memory because it doesn't build up a big array at all.

What's lighter than ExpressJS?

February 25, 2021
0 comments Node, JavaScript

tl;dr; polka is the lightest Node HTTP server package.

Highly unscientific but nevertheless worth writing down. Lightest here refers to the eventual weight added to the node_modules directory which is a reflection of network and disk use.

When you write a serious web server in Node you probably don't care about which one is lightest. It's probably more important which ones are actively maintained, reliable, well documented, and generally "more familiar". However, I was interested in setting up a little Node HTTP server for the benefit of wrapping some HTTP endpoints for an integration test suite.

The test

In a fresh new directory, right after having run: yarn init -y run the yarn add ... and see how big the node_modules directory becomes afterward (du -sh node_modules).

The results

  1. polka: 116K
  2. koa: 1.7M
  3. express: 2.4M
  4. fastify: 8.0M

bar chart

Conclusion

polka is the lightest. But I'm not so sure it matters. But it could if this has to be installed a lot. For example, in CI where you run that yarn install a lot. Then it might save quite a bit of electricity for the planet.

The best and simplest way to parse an RSS feed in Node

February 13, 2021
0 comments Node, JavaScript

There are a lot of 'rss' related NPM packages but I think I've found a combination that is great for parsing RSS feeds. Something that takes up the minimal node_modules and works great. I think the killer combination is

The code impressively simple:


const got = require("got");
const parser = require("fast-xml-parser");

(async function main() {
  const buffer = await got("https://hacks.mozilla.org/feed/", {
    responseType: "buffer",
    resolveBodyOnly: true,
    timeout: 5000,
    retry: 5,
  });
  var feed = parser.parse(buffer.toString());
  for (const item of feed.rss.channel.item) {
    console.log({ title: item.title, url: item.link });
    break;
  }
})();


// Outputs...
// {
//   title: 'MDN localization update, February 2021',
//   url: 'https://hacks.mozilla.org/2021/02/mdn-localization-update-february-2021/'
// }

I like about fast-xml-parser is that it has no dependencies. And it's tiny:

▶ du -sh node_modules/fast-xml-parser
104K    node_modules/fast-xml-parser

The got package is quite a bit larger and has more dependencies. But I still love it. It's proven itself to be very reliable and very pleasant API. Both packages support TypeScript too.

A particular detail I like about fast-xml-parser is that it doesn't try to do the downloading part too. This way, I can use my own preferred library and I could potentially write my own caching code if I want to protect against flaky network.

Sneaky block-scoping variables in JavaScript that eslint can't even detect

February 3, 2021
0 comments JavaScript

What do you think this code will print out?


function validateURL(url) {
  if (url.includes("://")) {
    const url = new URL(url);
    return url.protocol === "https:";
  } else {
    return "dunno";
  }
}
console.log(validateURL("http://www.peterbe.com"));

I'll give you a clue that isn't helpful,


▶ eslint --version
v7.19.0

▶ eslint code.js

▶ echo $?
0

OK, the answer is that it crashes:

▶ node code.js
/Users/peterbe/dev/JAVASCRIPT/catching_consts/code.js:3
    const url = new URL(url);
                        ^

ReferenceError: Cannot access 'url' before initialization
    at validateURL (/Users/peterbe/dev/JAVASCRIPT/catching_consts/code.js:3:25)
    at Object.<anonymous> (/Users/peterbe/dev/JAVASCRIPT/catching_consts/code.js:9:13)
...

▶ node --version
v15.2.1

It's an honest and easy mistake to make. If the code was this:


function validateURL(url) {
  const url = new URL(url);
  return url.protocol === "https:";
}
// console.log(validateURL("http://www.peterbe.com"));

you'd get this error:

▶ node code2.js
/Users/peterbe/dev/JAVASCRIPT/catching_consts/code2.js:2
  const url = new URL(url);
        ^

SyntaxError: Identifier 'url' has already been declared

which means node refuses to even start it. But it can't with the original code because of the blocking scope that only happens in runtime.

Easiest solution


function validateURL(url) {
  if (url.includes("://")) {
-   const url = new URL(url);
+   const parsedURL = new URL(url);
-   return url.protocol === "https:";
+   return parsedURL.protocol === "https:";
  } else {
    return "dunno";
  }
}
console.log(validateURL("http://www.peterbe.com"));

Best solution

Switch to TypeScript.

▶ cat code.ts
function validateURL(url: string) {
  if (url.includes('://')) {
    const url = new URL(url);
    return url.protocol === 'https:';
  } else {
    return "dunno";
  }
}
console.log(validateURL('http://www.peterbe.com'));

▶ tsc --noEmit --lib es6,dom code.ts
code.ts:3:25 - error TS2448: Block-scoped variable 'url' used before its declaration.

3     const url = new URL(url);
                          ~~~

  code.ts:3:11
    3     const url = new URL(url);
                ~~~
    'url' is declared here.


Found 1 error.

useSearchParams as a React global state manager

February 1, 2021
0 comments React, JavaScript

tl;dr; The useSearchParams hook from react-router is great as a hybrid state manager in React.

The wonderful react-router has a v6 release coming soon. At the time of writing, 6.0.0-beta.0 is the release to play with. It comes with a React hook called useSearchParams and it's fantastic. It's not a global state manager, but it can be used as one. It's not persistent, but it's semi-persistent in that state can be recovered/retained in browser refreshes.

Basically, instead of component state (e.g. React.useState()) you use:


import React from "react";
import { createSearchParams, useSearchParams } from "react-router-dom";
import "./styles.css";

export default function App() {
  const [searchParams, setSearchParams] = useSearchParams();

  const favoriteFruit = searchParams.get("fruit");
  return (
    <div className="App">
      <h1>Favorite fruit</h1>
      {favoriteFruit ? (
        <p>
          Your favorite fruit is <b>{favoriteFruit}</b>
        </p>
      ) : (
        <i>No favorite fruit selected yet.</i>
      )}

      {["🍒", "🍑", "🍎", "🍌"].map((fruit) => {
        return (
          <p key={fruit}>
            <label htmlFor={`id_${fruit}`}>{fruit}</label>
            <input
              type="radio"
              value={fruit}
              checked={favoriteFruit === fruit}
              onChange={(event) => {
                setSearchParams(
                  createSearchParams({ fruit: event.target.value })
                );
              }}
            />
          </p>
        );
      })}
    </div>
  );
}

See Codesandbox demo here

To get a feel for it, try the demo page in Codesandbox and note has it basically sets ?fruit=🍌 in the URL and if you refresh the page, it just continues as if the state had been persistent.

Basically, that's it. You never have a local component state but instead, you use the current URL as your store, and useSearchParams is your conduit for it. The advantages are:

  1. It's dead simple to use
  2. You get "shared state" across components without needing to manually inform them through prop drilling
  3. At any time, the current URL is a shareable snapshot of the state

The disadvantages are:

  1. It needs to be realistic to serialize it through the URLSearchParams web API
  2. The keys used need to be globally reserved for each distinct component that uses it
  3. You might not want the URL to change

That's all you need to know to get started. But let's dig into some more advanced examples, with some abstractions, to "workaround" the limitations.

To append or to reset

Suppose you have many different components, it's very likely that they don't really know or care about each other. Suppose, the current URL is /page?food=🍔 and if one component does: setSearchParams(createSearchParams({fruit: "🍑"})) what will happen is that the URL will "start over" and become /page?fruit=🍑. In other words, the food=🍔 was lost. Well, this might be a desired effect, but let's assume it's not, so we'll have to make it "append" instead. Here's one such solution:


function appendSearchParams(obj) {
  const sp = createSearchParams(searchParams);
  Object.entries(obj).forEach(([key, value]) => {
    if (Array.isArray(value)) {
      sp.delete(key);
      value.forEach((v) => sp.append(key, v));
    } else if (value === undefined) {
      sp.delete(key);
    } else {
      sp.set(key, value);
    }
  });
  return sp;
}

Now, you can do things like this:


onChange={(event) => {
  setSearchParams(
-    createSearchParams({ fruit: event.target.value })
+    appendSearchParams({ fruit: event.target.value })
  );
}}

See Codesandbox demo here

Now, the two keys work independently of each other. It has a nice "just works feeling".

Note that this appendSearchParams() function implementation solves the case of arrays. You could now call it like this:


{/* Untested, but hopefully the point is demonstrated */}
<div>
  <ul>
    {(searchParams.getAll("languages") || []).map((language) => (
      <li key={language}>{language}</li>
    ))}
  </ul>
  <button
    type="button"
    onClick={() => {
      setSearchParams(
        appendSearchParams({ languages: ["en-US", "sv-SE"] })
      );
    }}
  >
    Select 'both'
  </button>
</div>

...and that will update the URL to become ?languages=en-US&languages=sv-SE.

Serialize it into links

The useSearchParams hook returns a callable setSearchParams() which is basically doing a redirect (uses the useNavigate() hook). But suppose you want to make a link that serializes a "future state". Here's a very basic example:


// Assumes 'import { Link } from "react-router-dom";'

<Link to={`?${appendSearchParams({fruit: "🍌"})}`}>Switch to 🍌</Link>

See Codesandbox demo here

Now, you get nice regular hyperlinks that uses can right-click and "Open in a new tab" and it'll just work.

Type conversion and protection

The above simple examples use strings and array of strings. But suppose you need to do more more advanced type conversions. For example: /tax-calculator?rate=3.14 where you might have something that needs to be deserialized and serialized as a floating point number. Basically, you have to wrap the deserializing in a more careful way. E.g.


function TaxYourImagination() {
  const [searchParams, setSearchParams] = useSearchParams();

  const taxRaw = searchParams.get("tax", DEFAULT_TAX_RATE);
  let tax;
  let taxError;
  try {
    tax = castAndCheck(taxRaw);
  } catch (err) {
    taxError = errl;
  }

  if (taxError) {
    return (
      <div className="error-alert">
        The provided tax rate is invalid: <code>{taxError.toString()}</code>
      </div>
    );
  }
  return <DisplayTax value={tax} onUpdate={(newValue) => { 
    setSearchParams(
      createSearchParams({ tax: newValue.toFixed(2) })
    );
   }}/>;
}