Episode 142: Nebula Logger with Jonathan Gillespie

In keeping with our discussion of open-source software, today we will be speaking with Jonathan Gillespie, a software engineer here at Salesforce who is responsible for building the Nebula Logger. Jonathan is a proponent of open-source software for its many benefits, from community sourcing to expert knowledge and transparency.

Frustrated with the limitations of proprietary software, Jonathan began experimenting with open-source as a more robust alternative. In my discussion with Jonathan, he fills us in on the reasons he has embraced open-source and why it can be helpful to other developers. He also discusses his current project, Nebula Logger, what led him to begin his work on it, and how it can be used to monitor and report logging data.

Show Highlights:

What led Jonathan to embrace open-source software
The benefits of open-source software
The origin story of Jonathan’s work on Nebula Logger
How the rise of automation has impacted developers

Links:

Jonathan on Linkedin – https://linkedin.com/in/jongpie/
Jonathan on Github – https://github.com/jongpie/

Episode Transcript

Josh Birk:
That is Jonathan Gillespie, lead software engineer here at Salesforce. I’m Josh Birk, your host for the Salesforce Developer podcast. Here in the podcast, you’ll hear stories and insights from developers for developers. Today we sit down and talk with Jonathan about a longstanding open source project he’s been running, Nebula Logger. You might have heard it in our last week’s episode with James Simone. But we’ll start, as we often do, with his early years.

Jonathan Gillespie:
I think I did pretty early on want to get into computers. I didn’t really know what exactly I wanted to do. But I think around seventh, eighth grade, we got one of our first computers at home, AOL dial up. Very exciting time, really fell in love with tinkering on computers all day. Broke Windows 95 and Windows 98 once or twice. We had to reinstall. But I got into it very quickly.

Josh Birk:
I would ask how you break Windows 95, but I think just using Windows 95 can get you there. Is that right?

Jonathan Gillespie:
That is one way to do it, yes.

Josh Birk:
My next question was your earliest memory with a computer. Did you have something before the AOL days?

Jonathan Gillespie:
We did. We had a really old Apple Mac II, black screen with green text on it, all that good stuff, the giant floppy disk. We got that when that was already considered pretty old. It wasn’t the most exciting thing, but I did tinker around with that a little bit. We had a couple games on it. But it wasn’t until a couple years later when we got an actual, more modern at the time, computer that I really started getting into it.

Josh Birk:
Nice. When did you first encounter Salesforce?

Jonathan Gillespie:
I first encountered Salesforce when I was working for a nonprofit around 2011 or 2012. I honestly can’t remember exactly when I first started using Salesforce. I think it was end of the end 2011, early 2012.

Josh Birk:
It was the old time. Everything’s pre-pandemic at this point.

Jonathan Gillespie:
At this point, pre-pandemic. It was a decade ago basically. It’s been awhile.

Josh Birk:
Nice. How did you end up getting a job at Salesforce?

Jonathan Gillespie:
Lot of hops along the way. I’ve been doing Salesforce stuff for 10 years since that nonprofit job. I’ve done a little bit of everything since then, working in-house for Salesforce teams, doing some consulting jobs for a couple of years, then most recently, I started at Salesforce because one of my old friends and colleagues, James [inaudible 00:02:27], started here a year and a half or two years ago now. Once he had started the job, after he was here for a few months, he reached out to me to see if I was interested. I ended up applying for his team and we’d been working together ever since.

Josh Birk:
Awesome. As an aside, that’s got to be cool being able to get a job with your friend and work with him for a couple years. That’s nice.

Jonathan Gillespie:
It’s really great. This is technically the third company we’ve been employed together at. But we’ve known each other since 2014 or so, kept in touch over the years. We’ve actually done a lot of open source work over the years as well. It’s great knowing your coworkers and getting to work with friends on a daily basis.

Josh Birk:
Nice. I have a really good friend, one of my best friends, Steve. If you look at our resumes, you would think that we just worked side by side with each other basically since we left college. It’s funny how not true it is. We’ve been at the same company. But over the years, we’ve only actually worked on two projects together. It’s just bizarre. But how would you describe your current role?

Jonathan Gillespie:
I’m a software engineer here. We basically build on platforms. We’re doing implementation of Salesforce for Salesforce. We work on a team that we manage, I think, nine or 10 different Salesforce applications or unlocked packages that are used by various Salesforce teams internally.

Josh Birk:
When did you start getting involved in open source software itself?

Jonathan Gillespie:
I think as far as actually making contributions, I started somewhere around 2016 or 2017. Prior to that, I had benefited a lot from using open source software, especially in the Salesforce world. In particular, there was a time in my career where I was not sure if I wanted to fully commit to becoming a Salesforce expert. Partially, I was frustrated with some of the tools, including the force.com ID, which is just fancy for basically Eclipse with some extra plugins. Never been a big fan of Eclipse myself personally, just always felt a little bloated. That being really the only officially supported offering at the time, struggled a little bit. But MavensMate was a great open source tool. I don’t think it’s being used or developed anymore, but great at the time, really big game changer, and really just gave me a way forward as far as doing development without feeling it was just this overly clunky utility that I had to use.

Josh Birk:
That was a weird time because before MavensMate, people were splitting their time between developer console because in some ways, it was faster just to jump in a developer console and get some stuff done, or having to go through the force.com ID, especially if you’re doing something like a deployment night or something like that. I remember having a conversation, I think in GitHub or something like that, with Joe Ferraro because I was having that frustration too. I started writing these batch files that basically just worked against the… It was just bash against the tooling API basically. Joe saw what I was doing. I was just posting it online every now and then. Joe’s like, “We have all these internal tools that we use at Maven.” I’m like, “Share them [inaudible 00:05:29].” Then the community jumped on it. It’s a great example of the community suddenly coming together and being like, “Oh my God, we can really build something bigger, better, faster, and easier.” Didn’t it have a Pac-Man game or something while you’re doing your deploy, or is it Tetris?

Jonathan Gillespie:
It did, now that you mention it. It had some kind of old school arcade game built in.

Josh Birk:
Now we have… Wade Wagner really has a lot of credit to his name of coming in and trying to get our Salesforce DX back up and running. We push everybody over to Visual Studio now. But that was an interesting time. Did you use a lot of open source software while you were at the nonprofit?

Jonathan Gillespie:
Not as much then. We were a pretty big Microsoft shop at the time, so we used Microsoft’s SQL Server and .NET internally. But as we started getting into Salesforce and using things like Eclipse, we did start using Eclipse for some Java development as well internally and some other things like that. It varied. I wouldn’t say we were initially a huge open source shop, at least originally. It evolved some over time as we found open source tools that were useful.

Josh Birk:
Got it. Same question, but changing the time, how are you using it right now in your current role with the platform [inaudible 00:06:42] work?

Jonathan Gillespie:
We use open source pretty heavily at the moment. I think just the landscape for Salesforce development has changed over the last few years. Tools that are officially provided by Salesforce like SFDX are opensource themselves. We also use VS code, which I believe…

Josh Birk:
“I believe,” I know I’m having the exact same question in my head. I think VS Code is. In your current role, how are we using open source software with Salesforce itself?

Jonathan Gillespie:
We use open source a couple different ways. Some of it is just the landscape of Salesforce development these days. Tools like SFDX and VS code are being used pretty heavily, including by our team, all of which are open source. A lot of the tools that we use for doing things like writing code deployments and all that are open source already. Then we also use something I’m very excited about. We use internally the Nebula Logger project that I started several years ago and worked on for years with friend and colleague, James. We use that basically as a way to monitor our production instances for many errors for the applications that we maintain and support.

Josh Birk:
Nice. We’re going to dig into Nebula Logger, but just in general, what do you think are the pros and cons of having open source software as part of your daily routine, as something that’s core to your development cycle?

Jonathan Gillespie:
To me, I think it’s a great thing. I think everybody should consider using open source or at least look into projects that are open source to see if they would fit their needs. I think there’s just a lot of benefits in the community knowledge that goes into some of these projects. You have oftentimes experts that are working on some of this and just want to do their part to help out whatever development community they’re in. You get oftentimes people from different experiences and backgrounds that are all contributing and trying to make something very robust. I think there’s just the benefits of who’s creating the software itself. There’s also the benefit of, depending on the licensing, you oftentimes have full rights to reuse that code, make changes to it. It gives you at least a great starting point. I think there’s just a lot of benefits that people and companies can get from using open source software in their implementations.

Josh Birk:
I think some of the common things I hear some people say is it’s really secure if you can see all of the source code. In a conversation I was having with James, actually, in some ways it’s almost more secure because everybody can see all the source code. Do you find that trend, that the transparency is actually making more secure, more robust code?

Jonathan Gillespie:
I think so. I think security [inaudible 00:09:09] or basically when you can’t see the code, that doesn’t mean it’s secure. It just means you don’t know about it being insecure. I think to your point, it sounds like you and James discussed, I think it gives itself a better chance of being secure if you can actually look at the raw source code, see what’s happening, and help identify and maybe resolve any security holes.

Josh Birk:
I’m laughing. I’m going to love this piece. This is the first time I’m going to intentionally try to pair these two episodes together is. I think James actually may have used the exact same phrase. I love the logic though. Just because somebody doesn’t know that Jimmy broke the HTP code three years ago doesn’t mean that somebody’s going to figure it out. Security dev’s rife with all of these things.

Jonathan Gillespie:
Right.

Josh Birk:
Let’s dig into Nebula Logger itself. What’s the origin story there? What made you want to start working on this project?

Jonathan Gillespie:
It’s had a bit of a long history. It originally started, James and I both started working on a project that I called the Nebula Framework at the time. This was back in, I want to say end of 2016 or early 2017, somewhere in there. James and I originally worked at one company together in Boston. I had just moved to another company. We were still keeping in touch and still wanted to collaborate on things. He and I first set up our GitHub profiles and started collaborating with some stuff. We even started out trying to build basically just a whole framework we could use for Salesforce, things like trigger handler framework was involved, things for making match classes easier to implement and monitor.

Jonathan Gillespie:
Originally, it just started with less than 100 lines of code. I had the ability to also do some custom logging, which got stored in a custom object. It was very basic stuff. Originally, it was just a small subset of features within a much bigger trigger handler framework, which was part of a much bigger framework overall. James and I kept making enhancements to that for, I think, a couple of years. We made some progress on it. But at some point, especially as GitHub, GitLab, and Bitbucket all started offering more public and private repos, it was a lot easier to start sharing open source software, either selectively as a private repo, which didn’t exist in 2017, or we could just start making things more public.

Jonathan Gillespie:
As the Salesforce ecosystem started growing more and more, there were a lot more open source projects. Thus, there were also a lot more things like open source trigger handler frameworks. It just didn’t really make sense to put so much of my own time and effort into something that there were a dozen other projects doing a much better job of it anyway. I stopped doing work on the framework overall and instead started focusing just on the logging piece. That seemed to still be a gap in the ecosystem as far as there just weren’t a lot of great logging tools out there. I split it off into its own repository and have been working on it pretty heavily ever since, especially the last about two years. It’s really grown in terms of feature set as well as in popularity within the Salesforce ecosystem.

Josh Birk:
Nice. I love that. Going back to what we were talking about before, a benefit of open source from a community level, you could tell you were not spinning your wheels, but maybe reinventing the wheel a little bit. Then you could just reprioritize. But if everybody had just closed their software off, you’d probably still be working on the framework, right?

Jonathan Gillespie:
Absolutely. My goal wasn’t to keep job hopping over the years. But I think I’ve had five jobs in the Salesforce world over the last decade. It’s nice when you start a new job to still have a set of boiler plate code or a starting point, especially if you’re doing a brand new implementation. Had I not been able to find other people doing their own open source projects, I probably would’ve kept pouring all my own effort into maintaining my own framework. It’s a lot more reusable when people open source things.

Josh Birk:
Absolutely. Give me the elevator pitch for Nebula Logger. How would you describe it up and above just, “This is a logging system?”

Jonathan Gillespie:
My goal for the project is to basically make it the most robust logging tool for Salesforce. As the Salesforce landscape has changed over the years, what that exactly means has shifted a little bit. Three or four years ago, or even five years ago, people were focused more on things like Visual Force and Process Builder. Now it’s more around Flow and a little bit of [inaudible 00:13:27] components, but mostly Lightning Web components these days. My goal with the project is basically I try and make it a single unified logging system that works for whatever technology you’re planning to implement in your org, be it Flow, Apex, or lightning components, and just give you one centralized way to monitor and report on your logging data.

Josh Birk:
That’s a really interesting point. When you have your controller, your visual force, and you’re not really playing with a lot of other toys, maybe there’s a workFlow rule, validation rule, or something like that. But how is the rise of automation, background Flows, and things like that? Have you been reacting and having to adapt to that?

Jonathan Gillespie:
I think some of the older technologies, things like you mentioned, workFlow rules, they’re actually, I think, now officially on their way out in the next coming releases. Those are also just very limiting as far as trying to add any form of logging around those. But something like Apex, it’s code. It’s much easier to do things there. Then even with the shift in Process Builder to Flow, kudos to the Flow team for how they built things out already and how they continue to improve it because it is much easier to do things like invocable actions or calling Apex from Flows. It just really gives things a much easier way to do something or use something like Nebula Logger within these new types of automations that the platform supports.

Josh Birk:
Gotcha. How does that work from Nebula itself? When I’m logging things in Apex, do you have classes in Apex when I’m in Flow? Am I adding invocable actions in Flow in order to kick things off, or is there a different interface that I’m working with?

Jonathan Gillespie:
Most everything is written in Apex. If you look at the GitHub repository, I think it’s something like 98% of the code base is Apex Code. That works for both just Apex developers that want to do logging, they can call Apex classes directly, then there are also a couple of invocable Apex classes, specifically for use in Flow as well as Process Builder. The only part that’s not Apex right now is there is a Lightning Web component that you can use. You can embed it in either your own Lightning Web components or your own or components. Either way, it can also be used for front end logging.

Josh Birk:
Nice, like a nice system debug right there in your component.

Jonathan Gillespie:
Exactly. The nice thing, especially for front end development, is anything you do in JavaScript for console debugging, that only shows up on the end user’s computer. Doesn’t help you if you are an admin or developer who’s not sitting next to the person. Nebula Logger does solve that by actually saving the data into some custom objects that you can then report on within your org.

Josh Birk:
Very cool. Layering on top of that, what are some of your favorite features in Nebula?

Jonathan Gillespie:
That’s a great question.

Josh Birk:
I’ve been accused of asking people, “Which is the favorite of your children?”

Jonathan Gillespie:
Because of some of the historical context, I will say just the fact that you can do proper error logging on the platform is amazing and something you couldn’t do five or six years ago. In particular, platform events are a type of object that Salesforce has. Platform events are critical, I think, to making something like this work. Prior to platform events or if you’re not using platform events right now, if you try to say insert a record into a custom object and then some exception occurs later, an exception gets thrown, any kind of object you’ve tried to save to the database are going to get automatically rolled back by the platform. When James and I first started looking at this back in 2016, 2017, it was before platform events had been released. You couldn’t log something and throw an exception, which was very mind blowing for any developer coming from any other platform or background.

Josh Birk:
Right. A lot of times when I’m thrown into those situations, especially if it was… I remember being the team hero once just because I figured out how to throw something into a custom object because the user failed to get created. Then we would have a batch process, pick up the custom object, and go try again basically. But that got clunky over time. Was that similar to your original strategy, but now platform events gives you a faster, more robust, more real time framework?

Jonathan Gillespie:
That’s a big part of it. Some of what I was originally using logging for was we had some custom [inaudible 00:17:51] services within Salesforce. An external system would call these custom endpoints. Within that context, because it was all custom apex, it was a little bit easier to do things like log something into a custom object and give a response of some kind of error status code back to the external system. There were definitely some ways to get around these limitations, but it all was a little extra clunky and not really the ideal way for this to work.

Josh Birk:
Gotcha. How well does Nebula Logger work with external systems?

Jonathan Gillespie:
I think it works pretty well. I guess it depends on your exact use case. But there are different orgs using it now that have external systems that call and just do your basic create and read update operations on existing data. They have Nebula Logger integrated into their triggers, so they automatically get logging from that. There’s also ones that have still custom rest endpoints. They have Nebula Logger integrated there as well to get right at the rest endpoint level of code, getting extra logging in place.

Josh Birk:
Nice. That’s also an interesting tidbit, that it’s not just that you are contributing to this project, but people are also pulling in Nebula and using it as almost a logging standard. Does that sound right?

Jonathan Gillespie:
That’s what I’m aiming for. I think the project’s really grown a lot. I think I’ve seen a lot of usage over it over the last year or two. I’m really hoping to try and keep incorporating any feedback people will share with me and just try and make it basically the standard for logging.

Josh Birk:
How would you describe its current level of popularity?

Jonathan Gillespie:
At the moment, as far as I know, it is the most popular logging framework for Salesforce. I don’t have a whole lot, to be fair. I don’t have a whole lot to go off as far as how to prove that. But looking at things like number of commits or contributions to the repo on GitHub, the number of stars it’s received and forks that have been made, as well as just anybody that creates a second generation package, which is what Nebula Logger is or how it’s provided. There are ways within a dev hub in Salesforce to monitor how many times your package has been installed. Internally, I can also see I have hundreds of, or I think a couple thousand installs at this point.

Josh Birk:
Nice. That actually occurs to me, it being… Before second gen packaging, being able to be a logging… I don’t want to say logging as a service, but a logging package that could easily be installed in somebody else’s package and work nicely side by side before TGB. That would’ve been pretty darn difficult, right?

Jonathan Gillespie:
I think with the first generation packages, there’s just even more limitations around it. Some of those limitations still exist with second generation if you’re using a managed package. I do offer Nebular Loggers both an unlocked package without a name space, as well as a managed package. There’s definitely a few extra headaches that can come up with creating the managed package version, but pretty manageable these days.

Josh Birk:
How are you seeing adoption? Is this a purely developer play or is there an admin friendly route as well?

Jonathan Gillespie:
That’s another part of the project goals itself. I try and make sure that this is something that admins would want to use as much as a developer, as much as an architect. I try to make sure that the features I incorporate aren’t biased towards being better for an Apex developer versus a Flow builder doing admin declarative work. That seems to be working at the moment. There’s people that are just solo admin using it. There are people that have a whole team of admins or people doing declarative implementations. Then there are teams with multiple architects and multiple developers doing purely code written implementations. It seems to scale pretty well for everyone.

Josh Birk:
Got it. What else on the roadmap do you have?

Jonathan Gillespie:
Great question. I’m actually working on some new stuff this week. I think right now, things scale very well for if you were basically just doing a single implementation in Salesforce, if you just want to incorporate logging throughout. The parts that I’ve been getting a lot of feedback over the last couple months about are for the more complex orgs that either have a consulting company doing one piece of the implementation or multiple Salesforce teams doing different parts of the implementation and having a little bit more control over different, essentially modules within the code base to tailor logging a little bit more, is a gap that’s not there today. I’m working on some new features where basically if you have different modules that you’ll be able to define, you can adapt how exactly Nebula Logger behaves specific to your module. It just gives a lot more control to more complex teams.

Josh Birk:
Got it. Let’s start on the utilization side. If somebody wants to use this, where can they either install it or learn more about I?

Jonathan Gillespie:
Answer to both is go to the GitHub page or you can just go to Nebulalogger.com. That’s just going to redirect you to the GitHub page. On the GitHub page, it is the entire open source control for it. It also has links for installing both the unlocked package as well as the managed package.

Josh Birk:
How much from the community are you getting help on this so far? You were saying there’s tons of commits and accepting good old fashioned [inaudible 00:22:38] requests and stuff like that, or is this more of a John and James show?

Jonathan Gillespie:
Overall, it’s historically been mostly Jonathan and James show. There have been, over the last couple months, a lot more feedback as far as reporting issues or feature enhancements, which even that is a huge help. Then we’re also getting a handful of people that have been contributing code changes as well, which is also great.

Josh Birk:
Nice. If they want to learn it, they want to use it, they want to help develop it, Nebulalogger.com, and that will move over to the GitHub page.

Jonathan Gillespie:
That’s right.

Josh Birk:
Any other open source projects you’re working on right now?

Jonathan Gillespie:
I have a few other ones, nothing I’ve been doing as much lately. But I’ve been occasionally working on another one. I call it the custom metadata saver. It’s come in handy for a couple of people that have used it over the last probably six months. The idea is Apex does have the ability to modify custom metadata records. That actually kicks off a deployment, interestingly enough. I’ve basically wrapped some of that into some also invocable Apex actions. Now people can build custom UIs in Flow or LWC, let an admin actually modify custom metadata records directly, and then deploy them from there. It’s a little bit better user experience or can be a better user experience than the out of the box UI for custom metadata types.

Josh Birk:
That’s our show. But before we go, I did ask [inaudible 00:23:59] Jonathan’s favorite non-technical hobby.

Jonathan Gillespie:
Non-technical hobby. I just bought a house about two months ago now.

Josh Birk:
Congrats.

Jonathan Gillespie:
Thank you. Better or worse, whether or not I want it to be, my hobbies have now shifted to doing home repair a lot. That’s been great. There’s just a lot of little things we got to fix up all over the place, but I think that just comes with the territory of owning a home. Wife and I are super excited about it, keeps us busy.

Josh Birk:
Very cool. Double congratulations for surviving this particular real estate market.

Jonathan Gillespie:
Thank you.

Josh Birk:
It’s crazy out there.

Jonathan Gillespie:
For sure.

Josh Birk:
Jonathan, thank you so much for the conversation information. That was a lot of fun.

Jonathan Gillespie:
Awesome. Thanks so much for having me.

Josh Birk:
I want to thank Jonathan for the great conversation and information. As always, I want to thank you for listening. If you want to learn more about this show, head on over to developer.salesforce.com/podcast, where you can hear old episodes, see the show notes, and have links to your favorite podcast service. Thanks again, everybody. I’ll talk to you next week.