I had the unique opportunity of working on a project from start to finish, with complete independence. This was the first time I've had the opportunity to do this(apart from personal projects), and I was happy because I had the chance to try out a lot of interesting things.
Whenever I explore a new field I gain proficiency by exploiting the possibilities of that field and over-engineering the project to have some feel of the complexities involved.

Some Of My Horsemen

Horsemen

Some Project Requirements And Approach

Users should be notified about new/upcoming programs and resources => Backend, Push Notifications, Notifications Screen
There is no authentication but end users should be distinguishable from each other => Device ID
The API should be secured and only accessible by app users => Since there is no authentication I opted for a Shared JWT.
Users should be able to register and unregister for programs and the admin should be able to send notifications to users registered for a program => Push Notification Topics. Registered users are identified by the combination of the device ID and email used to register.
Users should have access to quarterly prayers and should be able to save them offline => Local Storage. Repository pattern with clean architecture.
Users should have a daily reminder of prayers for that day => Happens on clients using the alarm system.

About The Project

The project was for an International Christian Mission Organization, they had programs, resources, and prayers that they needed to share with their missionaries and interested users.

Workflow

Everything fell into their natural order based on requirements. These are the steps for engineering.

Briefing
Sketching
Low-Fidelity Design (Wireframing)
High-Fidelity Design and Prototyping
Frontend (Mock Backend)
Backend
Frontend-Backend integration
Tests and Cleanups
Package for Market
Ship Product

1. Briefing

The first step was to discuss with the project owner the needs of the organization and how the application could meet those needs. During that conversation, I recorded separate features and identified relationships between them. I then reviewed with the project owner what I had gathered from our conversation to ensure we were on the same page.

2. Sketching

After the briefing, I created around 20 pages of white paper mockups for features that could address the organization's needs. However, I realized that this was too many mockups and it was not aligned with the goal of creating a simple app. Since this was not my area of expertise, I sought advice from an experienced designer through ADPList. They advised me to reduce the mockups to only the essential elements that were necessary for the app's existence and start from there. Following their advice, I made the difficult decision to trim down my mockups.

3. Wire Framing

Coming from Adobe XD I knew it was high time I learned Figma so I took a 10-hour course on Figma Design and read almost all there was to read about Material 3. After that, I jumped straight into working on the Low fidelity design using the Material 3 design Kit for Figma.

4. High-Fidelity Design

To make the low-fi design pop, I had to go learn material styles. Typography, color, shape, and elevation

5. Frontend

Now to my domain. Jetpack compose straight up.

Fortunately for me Jetpack Compose supported Material 3 even though it was experimental at this time. It was a real beauty to see the material 3 design and jetpack composes implementation come together nicely.

I didn't like compose navigation at all especially passing data between screens, it was sort of like the same problem Flutter navigation 2 had all over again. So I looked out for other navigation options with a good sense for passing data between screens and I found Voyager I tried it out and it fit my need well except for a small issue which I chose to overlook since I didn't have a better alternative. Used Room for structured offline storage and Datastore for unstructured data storage. Since the Backend was not available I implemented the repository pattern and created a Mock Backend service at the end of the line.
Since the project is for an international organization there are plans to make use of Kotlin Multiplatform to support ios and implement Localization.

Quick Tip: As of the time of writing this article, Jetpack Compose Material 3 API does not support BottomSheets so I had to use Voyager's Bottomsheet Navigation

6. Backend

This was the main reason I was interested in this project, the opportunity to explore backend. So I had a few options:

NodeJs was a no-go because I knew Java script was absolute chaos, coming from an OOP background I had better chances with Typescript
Spring Boot was not an option either because It was Java and I had given up Java after seeing the light of Kotlin. Even though spring boot now supports Kotlin it still has too many annotations and boilerplate for my liking.
Ktor was my final choice because it was lightweight, and built with Kotlin. One more reason why kotlin was so perfect was its support for Domain Specific Language(DSL)

Next up I took a course on Ktor which just happened to use GraphQL so I started learning GraphQL. I used kGraphQL for the Kotlin implementation of GraphQL.

For the database and models, I used to kMongo to connect to MongoDB. Since I had already Mocked the backend on the mobile app, structuring the models was easy. The beauty was copying data classes from the backend to the frontend since they were both Kotlin.
After connecting everything I realized the response time was quite slow and that's when I discovered caching.

Caching

I researched java caching libraries and found caffeine which was quite good and seemed very fast. I decided to go with Redis because that was the popular choice. The next task was learning Redis and using it in a java environment so I went through the docs and chose kReds which seemed like it was perfect for kotlin. I worked with KReds through development but I changed it to lettuce at the point of deployment because kReds could not connect to the remote redis cache with a redis connection URL. The change was possible because I abstracted the caching implementation to a very low level.

I just realized that I created a caching DSL while writing this article 😂. This caching logic might be helpful for anyone interested. I can't explain it here but you can reach out if you need an explanation.

package org.capromissions.core.utils

import io.lettuce.core.api.sync.RedisCommands
import kotlinx.serialization.KSerializer
import org.capromissions.core.constants.EnVars
import org.capromissions.models.Model
import org.koin.core.component.KoinComponent
import org.koin.core.component.get
import org.litote.kmongo.coroutine.CoroutineCollection
import org.slf4j.LoggerFactory

object CacheController : KoinComponent {
    private val client: RedisCommands<String, String> = get()
    private val logger = LoggerFactory.getLogger("CacheController")

    suspend fun <T : Model> get(
        id: String,
        collection: CoroutineCollection<T>,
        serializer: KSerializer<T>,
        getData: suspend CoroutineCollection<T>.() -> T?
    ): T? {
        if (!EnVars.cacheEnabled) return getData(collection)
        val cacheKey by lazy { collection.namespace.collectionName }
        val data = client.hget(cacheKey, id)
        return if (data != null) {
            logger.info("CACHE HIT get $cacheKey")
            JsonConverter.fromJson(serializer, data)
        } else {
            logger.info("CACHE MISS get $cacheKey")
            val realData = getData(collection)
            set(collection, serializer, setData = { realData })
        }
    }

    suspend fun <T : Model> getAll(
        collection: CoroutineCollection<T>,
        cacheKey: String = collection.namespace.collectionName,
        serializer: KSerializer<T>,
        getData: suspend CoroutineCollection<T>.() -> List<T>
    ): List<T> {
        if (!EnVars.cacheEnabled)
            return getData(collection)
        val data = client.hvals(cacheKey)
        return if (data.isNotEmpty()) {
            logger.info("CACHE HIT getAll $cacheKey")
            data.map { JsonConverter.fromJson(serializer, it) }
        } else {
            logger.info("CACHE MISS getAll $cacheKey")
            val realData = getData(collection)
            setAll(collection, cacheKey, serializer, setData = { realData })
            realData
        }
    }

    suspend fun <T : Model> set(
        collection: CoroutineCollection<T>, serializer: KSerializer<T>, setData: suspend CoroutineCollection<T>.() -> T?
    ): T? {
        if (!EnVars.cacheEnabled) return setData(collection)
        val cacheKey by lazy { collection.namespace.collectionName }
        val realData = setData(collection)
        return if (realData != null) {
            val modelId = realData.id!!
            val response = client.hset(cacheKey, modelId, JsonConverter.toJson(serializer, realData))
            logger.info("CACHE SET set $cacheKey - $response")
            clearVolatile(collection)
            get(id = modelId, collection = collection, serializer = serializer) { null }
        } else null
    }

    suspend fun <T : Model> setAll(
        collection: CoroutineCollection<T>,
        cacheKey: String = collection.namespace.collectionName,
        serializer: KSerializer<T>,
        setData: suspend CoroutineCollection<T>.() -> List<T>?
    ): Boolean {
        if (!EnVars.cacheEnabled) return setData(collection) != null
        val realData = setData(collection)
        val map = realData!!.associate { it.id!! to JsonConverter.toJson(serializer, it) }
        if (map.isNotEmpty()) {
            val response = client.hset(cacheKey, map)
            logger.info("CACHE SET setAll $cacheKey - $response")
        }
        clearVolatile(collection)
        return true
    }

    suspend fun <T : Model, R : Any> getVolatile(
        fieldName: String,
        collection: CoroutineCollection<T>,
        serializer: KSerializer<R>,
        setData: suspend CoroutineCollection<T>.() -> R
    ): R {
        if (!EnVars.cacheEnabled) return setData(collection)
        val cacheKey = collection.volatileCashKey()
        val cache = client.hget(cacheKey, fieldName)
        return if (!cache.isNullOrEmpty()) {
            logger.info("CACHE HIT getVolatile $cacheKey")
            JsonConverter.fromJson(serializer, cache)
        } else {
            logger.info("CACHE MISS getVolatile $cacheKey")
            val realData = setData(collection)
            setVolatile(fieldName, collection, serializer, realData)
        }
    }

    private suspend fun <T : Model, R : Any> setVolatile(
        fieldName: String, collection: CoroutineCollection<T>, serializer: KSerializer<R>, setData: R
    ): R {
        if (!EnVars.cacheEnabled) return setData
        val cacheKey = collection.volatileCashKey()
        val response = client.hset(cacheKey, fieldName, JsonConverter.toJson(serializer, setData))
        logger.info("CACHE SET setVolatile $cacheKey - $response")
        return getVolatile(fieldName, collection, serializer) {
            setData
        }
    }

    fun <T : Model> CoroutineCollection<T>.volatileCashKey() = "${collection.namespace.collectionName}:volatile"

    private fun <T : Model> clearVolatile(collection: CoroutineCollection<T>) {
        client.del(collection.volatileCashKey())
        logger.info("CACHE CLEAR VOLATILE ${collection.volatileCashKey()}")
    }

    suspend fun <T : Model> cacheDel(
        id: String, collection: CoroutineCollection<T>, deleteData: suspend CoroutineCollection<T>.() -> Boolean
    ): Boolean {
        val cacheKey by lazy { collection.namespace.collectionName }
        return if (deleteData(collection)) {
            client.hdel(cacheKey, id)
            true
        } else
            false
    }
}

As you can see there are a lot of generics and recursion everywhere, and this became a problem when it came to deserialization as you have to provide a type or class. That is where I fell into one of the scariest errors I had encountered
-Inline Recursive Cycle-
It was so scary because I had come so far with the abstraction and I wasn't about to tear everything down because of a language limitation(If you understand the meaning of inline and recursion you'll understand why putting them together is an abomination). I had a similar problem with this StackOverflow question.
For the serialization and deserialization problem, I tried out a few options

Gson was my first choice because that was the default serialization choice of Ktor, but it didn't work out because it didn't accept generics and did not work well with inline functions.
Jackson was my next option but it did not work out because it couldn't handle deserializing generic collections
Enter Kotlinx Serialization which supports generic-type deserialization by enforcing kotlin serialization at compile time. This solved the issue the other libraries had. Apparently, kotlin losses reference to a generic type at run time which is why it didn't work. I was able to solve it with this

object JsonConverter {
    fun <T> fromJson(serializer: KSerializer<T>, json: String): T {
        return Json.decodeFromString(serializer, json)
    }

    fun <T> toJson(serializer: KSerializer<T>, value: T): String {
        return Json.encodeToString(serializer, value)
    }
}

Inspecting and debugging errors from the console was a headache because the console didn't have colors or stacktrace links like in mobile development's log cat so I integrated Sentry. Sentry has so much to offer and I'll recommend it to anyone. One thing I loved was their support for multiple environments.

Monitoring

After watching a documentary on the origin of Premethus I looked into monitoring options on Ktor and learned about how Grafana works well with it. I have to say that the whole Prometheus-Grafana relationship is a match made in heaven. It is just analytics for the backend.
This article was very helpful in setting up Grafana and Prometheus for Ktor

Deploying

My first option was Heroku because it was quite popular, but since they stopped offering free product plans I had to look for alternatives.

Google's App Engine was a great choice because it was dedicated to java environments the only issue I had with it was its support for environment variables and multiple work environments. At the time this was either nonexistent or difficult to find and implement.
Railway was such a great choice because it supported any language, supported docker, supported environment variables, and multiple environments. The possibilities were limitless, and their UI and UX were top-notch. So this was my final choice.

One of my proud stunts was shoving an entire JSON file into an environment variable 😎. Well, why did I do it? It was a secrete file 😎

7. Frontend-Backend integration

Now the main issue here was connecting with a graphql backend. I mean I have connected with a graphql backend before but that was with a rest client and with flutter 😂. I wrote the queries as String and sent them as a POST request, aah good old times 🤣.

This time was different because I found a graphql client called Apollo Kotlin. This was great because Apollo kotlin generates classes by introspecting your API so I didn't have to do any dirty work. That reminds me, I tried to connect my backend to the Apollo dashboard many times but it just didn't work. If anybody knows how that works with Ktor please let me know.

Since I had mocked the backend, I just plugged in the apollo client to the end of the line and it worked out(Not without a lot of hiccups).

Since the backend had two environments I created two build variants that pointed to the respective environments.

Quick Tip: During development, the android app was pointing directly to localhost and the port the server was running on (8080). To map the android devices port to the port of the laptop, I used this command

adb reverse tcp:8080 tcp:8080

I also encountered a dilemma regarding the implementation of notifications.

Notifications are sent from the backend to notify the user of new updates or information on a program the user registered for.
Notifications were also sent from the client to the client. i.e Daily prayer notifications from the prayers stored in the room database.

The problem here was "Where will the notifications be stored".
If they were stored on the backend then the database will get bloated and out of control or I'll have to do routine cleaning. Since that was not going to work, I decided that the notifications will all be stored on a local database on each client. That was notifications from both the client and backend will fall into the same bucket.

8. Tests and Cleanups

Unfortunately, like all the bad engineers I didn't settle down to do UI and unit tests because I was on a schedule to deliver before the start of the year. I just did multiple manual tests, some manual tests even failed 😂. Fixed what we could fix and moved on, besides it is an MVP.
I found a cool test UI testing framework called Maestro. That would be my first option when am ready to include tests.

9. Package for Market

This stage is like wrapping up a present, I worked on designing the launcher Icon in Figma

The rest was just privacy policies, product descriptions, and other content for the app store. For the app store screenshots, I felt very lazy so I just used Instamocks to create screenshots for Play Store.

10. Shipping

Finally the last stage. Surprisingly play store requested Identity verification before publishing the app. Since this was for an organization I had to wait to get the correct documents for verification.

Our verification attempts were rejected multiple times because the organization documents submitted were old. After some research, I realized what I did wrong when I created the developer account. I selected "Organization" as the account type, now this will also make your google payments profile account type to be "Organization" and that will require you to verify your identity and organization identity before you are allowed to publish.

Note: If you want peace of mind, just choose individual as

We were eventually verified and the app was published. Thanks for following me through this journey, I hope you gained something or at least gained motivation to try something out of your comfort zone.

You can check out the final product here and give your reviews on the experience.

New Things Learnt

Ktor
MongoDb
Redis Caching
Serverside Push Notifications
Amazon S3
GraphQl
Navigation with Volley
Docker and Docker Compose
Deployment with Railway.app, Heroku and Google's App Engine
Jetpack Compose Material 3
Figma Material 3, UI Sketches and Wireframing
Monitoring With Prometheus and Grafana

Bonus Section - Software Architecture

I explored software architecture, watched a few videos and realized I liked it. It solves the problem I always thought about when I was briefed on a project or onboarded on a team. It is a great solution to help business executives have an idea of how their technology works. It is also great for onboarding new members of a team.

I checked a lot of options some of which are Luchid charts and Draw.io. These were great but I came to understand that these were general-purpose diagramming software and not dedicated to Software Architecture.

Finally, I found Icepanel.io which was the absolute best of my options. I always wanted to display a high-level view of a software system and a low-level technical part. I also wanted to show relationships between components and explain flows used to perform a task. Icepanel did all this and also explained how their modeling system is based on the C4 Model.

In retrospect, I believe that right after design and prototyping every engineering project should start with a software architect scoping out the relationships and possibilities, and limitations of the system. Just like wireframing for UI

Case Study 1: Full Stack Engineering - Figma, Android, Ktor

Start To Finish

Table of contents