mobile local web-server to store local data by a web app
#help
Nando

Nando @fc250152

About: an old developer, now retired but still interested in software. currently studying a project for enterprise app generation from declaratives.

Location:
Italy
Joined:
Jan 31, 2018

mobile local web-server to store local data by a web app

Publish Date: Apr 25 '19
6 13

Hi all

scenario:
last year I've developed a web app for android devices that should work off-line.
The app serves as front-end of the main app running on the main web server.
When on-line, the app should receive from the main web server the data about specific tasks that should be done off-line.
When again on-line, the app should return to the main web server the new data of the completed tasks.
In order to work off-line the app stores data in the internal storage of the browser (local-storage or indexedDB).
The application is cached in the Application Cache of the browser (according to the supplied manifest file).
I've not developed a native app for several reason, that cannot be overcame.

new scenario:
the now mandatory use of https does create problems with AppCache; furthermore, the use of the browser internal store seems too limiting and the browser sandbox in general doesn't allow complete operational freedom.
I'm thinking about a solution based on having a web server running locally (on the android device). The web app could talk with a simple server app in order to store and retrieve data in a relational db, and also to carry out other operations such as documents generation and likes.

questions:
what do you think about this, guys?
do you have any hints?
do you know any product (free) that may be used?
maybe a NodeJS running on Android?

I would warmly thank you in advance, guys.

last but not least: please excuse my poor and bad english :(

Comments 13 total

  • Jaaki
    JaakiApr 25, 2019

    Personally I find that the in browser storage is really adequate for just about anything. Local and Session is what I use most and being key->value stores, you can do much with it.

    I did build a solution based on storing "tasks" locally when offline, and when the browser goes online, would run through those task sending them to the main server.

    These days I've become spoilt with using Firebase's Firestore NoSQL backend as it handles all the offline support right out of the box, nothing extra to be done by you. It just works. (Free for small projects)

    You could also look at some abstraction packages for the browser localstores, something like LocalForage comes to mind. It handles json and also blobs amongst other types.

    • Nando
      NandoApr 25, 2019

      Thank you for your reply, Jaaki.

      I've already used LocalForage and I got the job done.

      But now I'm searching for a wider solution to overcome the various limits imposed by the browser (they are there for security reasons, of course).
      So I'm looking for a solution where the app in the browser could talk with a local process acting as a web server running in the Android device. In this way all operations could be possible, including reading/writing files, access any relational db (SQL), etc...
      I'd like such process was a NodeJS instance :)

      However, I'm very grateful for your response.
      God bless

      • Jaaki
        JaakiApr 25, 2019

        I forgot to mention PouchDB, that's nice.
        It is an auto synchronising db, but if you really need SQL, then...

        It sounds like you are looking for something like WebSQL, but that has been deprecated due to difficulties in standardising, but it still seems to work across many devices.

        For reading and writing files, Filesystem-API will work. You can request permission to store files on the client.

        Maybe you should look at what you can do with a ServiceWorker to proxy the outgoing connection.

        So, ServiceWorker that writes to WebSQL and FileSystem sounds like what you are after.

        • Nando
          NandoApr 26, 2019

          I would thank you for your kind help, Jaaki; it's much valuable.
          I've used some of the solution you propose, except ServiceWorker; it is not so plain for me, I don't "feel" it :( furthermore PWAs in general mandate the use of https, so certificates and so on..
          But I ultimately would escape the browser world, 'cause it currently has severe limitations (also in Filesystem API, which I've tried) and more of them may come in the future due to security reasons.
          So I'm looking for a solution like the one that I've outlined in my original post as "new scenario".
          Do you have any hints on it? (but maybe you are a front-end man and not a full-stack one ...)
          Thanks again, Jaaki. Have nice days.
          Nando

          • Jaaki
            JaakiApr 26, 2019

            Service workers basically replaced AppCache.
            I think you should take a look at them in detail as they are a major component in the offline system.

            "New Scenario" looks like you want a PWA plus a native app (local server) in the background to bypass security constraints of the browser system. Bad idea!

            Maybe, if you want the PWA to act like a native app, it should in actual fact BE a native app, but it is hard for me to suggest what you should do because I don't understand exactly what you want to do and whether the Filesystem-api etc. is in fact to limiting for what you are trying to accomplish.

            • Nando
              NandoApr 26, 2019

              Hi Jaaki.
              I think that a native app can do anything it is needed without suffering of limitations. It's not so for a web app, due to (legitimate) browser constraints.
              If I could develop a native app I could do things that I'm not able to do in the browser, right? So, I thought it was possible to combine the two: where is the bad?
              However, I will try in some other way; I've found an interesting app named AndroidJS; maybe it's a possible solution.
              Thanks again for your help
              God bless

              • Jaaki
                JaakiApr 26, 2019

                Yes, PWA runs in a sandbox, probably always will.
                Native app is much more powerful.

                You should then look at the new Ionic 4 framework. It is a favourite for many developers who are testing new products. It now allows you to use much of your existing Vue app inside a Hybrid environment. You can add plugins like SQlite, but then you have to deploy it through the AppStore/Google Play, so you need to pay developer fees etc.

                A quick overview:

                PWA (browsers only), runs in limited sandbox, but is easy to deploy. (Push to webserver takes 2 minutes for update)

                Native App, most powerful with best performance. Deploy through AppStores/PlayStore, takes long to update.

                Hybrid Apps, sits somewhere in between. Performance not so good, but getting better. If you want native plugins like bluetooth and sensors or SQLite, you have to deploy to AppStore/PlayStore.
                If you build a basic system, (sand boxed browser), deploy to webserver is possible.

                I had some experience with Cordova, but the apps were just painfully slow.

                • Nando
                  NandoApr 26, 2019

                  Hi Jaaki,
                  I've read carefully your explanation.
                  The current version of my app is of the Hybrid type, then it suffers of running in the browser sandbox, but may use AppCache and LocalStorage.
                  The major problem I have is caused by the mandatory use of https in the last versions of browsers; https doesn't accept the use of the AppCache :(
                  I don't absolutely like the use of ServiceWorkers as workaround for the AppCache.
                  This fact, togheter with other possible future limitations imposed by the browsers, pushed me to search an alternative solution as described in "new scenario".
                  I would like to find a way to proxy to a local app the operations that cannot be executed natively in the browser or are better executed in background.
                  I would hearthly thank you for your kind help, Jaaki.
                  Have nice days

  • rhymes
    rhymesApr 27, 2019

    Hi Nando, I agree you should move away from the AppCache, mostly because it has been deprecated and it's going to be removed.

    A possible alternative for offline usage are service workers as Jaaki suggested, though I admit they are more complicated to use. A few resources:

    So this might be option 1, re-implementing the AppCache code with Service Workers.

    Option 2 might be using something like AWS AppSync that lets you sync your app for offline usage:

    AWS AppSync supports an offline programming model where application data is not only available offline, but users can add and update data as well. When the device is offline, the application UI will be automatically updated with the offline data. AppSync lets you define how data is cached offline as well as how AppSync should manage the cache updates under different network conditions

    Option 3, as you suggested, would be embedding a web server, like nanohttpd I guess but how do you sync data from the local copy to the upstream server? What happens if the local web server process crashes? Do you have any guarantees? I'm not familiar with Android development so I might be raising problems that are not problems indeed.

    Regarding storage: if you use a service worker you can use IndexedDB through localForage or PouchDB, if you use AppSync they store your data (I believe it's inside Amazon DynamoDB) remotely and locally, if you use an embedded webserver you can probably use Android's own DB storage libraries.

    Hopefully this helps clear some doubts

    • Nando
      NandoApr 27, 2019

      Hi friend, thank you so much for your comprehensive and exhaustive explanation.
      I will study carefully each of the options you have exposed.

      Only to better define the scenario already in place:

      • the app in the browser receives from the main web server one or more "documents" to complete offline, then transmits them back to the same server when they are ok.
      • in order to work offline, the app uses the AppCache for the pages and the LocalStorage to store the data.
      • the data are stored in a different format with respect to the one that they have in the main server: an "internal" format, known only to the front-end app.

      In the "new scenario" I would give up both the AppCache and the LocalStorage, and possibly also the other limitations imposed by the browser by "proxing" data operations to a local (pseudo)webServer.
      That's all.

      Many thanks for your help!
      Have nice days
      Nando

      • rhymes
        rhymesApr 27, 2019

        I think you could go incremental with this thing. First use localForage or a similar option so that you can abstract from localStorage and maybe use IndexedDB. Then you can look into replacing AppCache with Service workers.

        If that it's not enough then you can write all the code to proxy the webserver.

        I'm just saying this because that option seems the most complicated and the one with the most code, while you can maybe settle to not having a proxy at all.

        Have fun!

  • s10ford
    s10fordMay 22, 2019

    Dear Nando,
    maybe this project is something that can be useful for you: paw-android.fun2code.de/
    It is a local webserver for Android that supports https. BeanShell scripts can be used to access native functions of your phone, like accessing the filesystem, read out sensors, etc. Source code is available on the website.

    • Nando
      NandoMay 22, 2019

      Hi bro, thank you very much for your suggestion!
      It seems to fit perfectly, but I have to take some time for studying it well.
      Thanks again,
      God bless

Add comment