Verrency is one of Melbourne's most exciting fintech start-ups.
The Verrency platform allows banks to include value-added services to their payment pipeline without a significant internal IT spend and provides solutions to personalize aspects of the banking experience so consumers can tailor products to their own needs.
They use Apache Kafka® to power these card and payment innovations and are early adopters of Kpow.
Growing a high-performing team in a cutting-edge space has unique challenges. We spoke to CTO Euan Walker about the journey so far.
A Blank Slate
Fintech is a juxtaposition of technical innovation and a conservative market. The Verrency platform is trusted by Financial Institutions and Partners worldwide. They expect zero downtime and real-time performance while often requiring the ability to consume data at their own pace.
Development of the platform began in 2016 with delivery and maintenance managed by a small team. Core technology choices and engineering empowerment are critical success factors for any software project but even more so when you're a start-up balancing performance targets with operational flexibility.
Euan explains, "As a fintech focused on global growth and working with some of the most conservative technology buyers (banks), we constantly have to balance the need to be operationally flexible. We do this while having a zero downtime goal and hitting performance targets that don't leave a lot of room for mistakes without gold plating the solution."
One architectural challenge faced by the team was identifying a suitable technology stack for modern payment processing. Verrency's prior experience in banking and finance gave confidence that the JVM was the correct bedrock - but innovation requires risk, and real-time, reliability, and durability requirements called for a new approach.
"We've been fortunate to have engineers who know Kafka well, but it has a learning curve."
Verrency chose to adopt Kafka as the heart of real-time computation, as a durable log of transaction histories, even as an integration option for partners and third-party providers. To the team, this represented a learning curve with an unfamiliar, rapidly evolving technology. Early successes built confidence, and the framework soon became mission-critical to the success of their platform.
It's Not Easy Being Early
Verrency was one of the first teams to adopt Kafka as a full streaming compute platform in Melbourne. "We started working with Kafka four years ago -- initially as a pilot where we were also looking at Kinesis, and then as a full part of our infrastructure," says Euan. "There still aren't many people who've done big, complex Kafka projects here in Melbourne, and that can pose a challenge."
The challenge was scarce opportunity to share and learn with peers and plenty of time grinding through Kafka Streams issues or learning RocksDB idiosyncrasies. Over time that has changed, and Verrency has become a core part of the Melbourne Kafka scene, participating and contributing to this growing community through the Melbourne Distributed Meetup, which our CEO and Verrency's former head of engineering run together.
The Verrency team has learned plenty of lessons since adopting Kafka. Some of the hardest came in operations. For a seemingly simple distributed log, managing Kafka Clusters in production requires experience and dedicated resources.
"Kafka is powerful when used well, though there are challenges. There isn't a particularly rich or cost-effective eco-system of tools to help with management, deployment, and monitoring - this has started to change in the last 12 months with the like of Amazon's MSK, and tools like Kpow"
Amazon MSK became a vital part of the mix, reducing the operations surface area of Apache Kafka and allowing a product-focused fintech start-up to spend time and talent building products rather than spelunking logs.
Ghost in the Machine
Kafka introduces terms that can be bewildering to traditional engineering teams who move into streaming compute. Kafka Streams adds to the cognitive load by introducing even more concepts. Among the topologies, partitions, brokers, and offsets, Euan highlights an age-old problem for any programmer - where is my data? "With Kafka, your data is hidden away, and troubleshooting can be time-consuming -- even for what you hope would be a simple task, I think this is the key challenge for Kafka in smaller organizations," says Euan.
Data Inspect has long been the most popular feature of Kpow, in part because Verrency engineers were such vocal beta testers and their feedback played a prominent role in shaping the product development. We thrive on feedback we get from our users, and our soon to be released kREPL and custom Kafka Language are other examples of features that incorporate requirements from engineers at practice.
"Kpow is built by a team with a passion for Kafka, but who also carry the scars of some tough implementations. It's a tool by and for engineers, and I look forward to seeing it continue to evolve."
For Verrency, using the Kpow Kafka toolkit is a no brainer, as it offers their engineers the simplest, quickest, most cost-effective way to access their data. Even more so as it works seamlessly with Amazon MSK, Euan says that 'Kpow, will be a key part of our infrastructure as long as we're using Kafka."
Verrency has recently been named a finalist @ausfintech's Finnies Awards for Excellence in Payments and Excellence in Establishing Global Presence. They have a growing team and an exciting market-leading product. For more information about the Verrency Platform, head to verrency.com or request a demo.
Manage, Monitor and Learn Kafka with Kpow by Factor House.
We know how easy Apache Kafka® can be with the right tools. We built Kpow to make the developer experience with Kafka simple and enjoyable, and to save businesses time and money while growing their Kafka expertise.
Kpow is compatible with Apache Kafka+1.0, Amazon MSK, Instaclustr, Aiven, Red Hat AMQ Streams, Azure Event Hubs, Confluent Platform, and Confluent Cloud.
Start with a free 30-day trial and solve your Kafka issues within minutes.
Kylie Troy-West is a Co-Founder and COO of Factor House. Her focus is empowering the Factor House team to deliver their vision of world-class tools for Apache Kafka.