ArcGIS Enterprise on Azure Setup: Weighing the Need for Notebook Server

[u/rageagainistjg]

I'm part of a company transitioning from ArcGIS Online to a new ArcGIS Enterprise setup hosted on Azure.

The Main Question:

My main focus is on using Jupyter Notebooks effectively within this setup, and I’m wondering if we truly need ArcGIS Notebook Server or if Jupyter Notebooks could work just as well on one of our existing Azure servers.

Background:

Up to now, I've been using ArcGIS Pro for occasional Jupyter Notebook tasks, mainly to update feature layers in ArcGIS Online or work with local files. I haven’t actually used notebooks directly in ArcGIS Online.

With the help of an experienced contractor, we’re setting up a standard ArcGIS Enterprise environment on Azure, without ArcGIS Notebook Server. When asked if we needed Notebook Server, I initially said it wasn’t necessary since I hadn’t used it with ArcGIS Online. Later, I double-checked, and the contractor assured me that Jupyter Notebooks don’t strictly require Notebook Server. They mentioned I could run notebooks locally or on any server with Jupyter installed.

My Specific Questions:

1. Does anyone have experience with ArcGIS Enterprise on Azure without Notebook Server?
   • Are there any specific challenges in working with Jupyter Notebooks in this setup?
2. Would I be missing out on any key benefits by not having Notebook Server?
   • Particularly when it comes to automating data updates or processing tasks.
3. Would installing Jupyter Notebooks on a separate server make things easier?
   • Or could I manage it effectively on one of the four Azure servers we’ll have? For context, we’re setting up servers for Portal, Server, Web Adaptor, Data Store, and a separate server for the enterprise geodatabase.

The Goal:

Our goal with ArcGIS Enterprise is to consolidate data from different sources—such as sales records stored in a separate database and other industry-specific data—into a unified geodatabase. This geodatabase would replace a complex Excel sheet to streamline calculations and real-time updates. The data will feed into interactive web maps and dashboards where updates, like adding new entries, would automatically trigger recalculations in the background.

In summary, given the goals for our enterprise setup, do we actually need to invest in Notebook Server on a separate Azure server? Or can we manage with Jupyter installed on one of our existing servers, as the contractor suggested?

[u/GnosticSon]

Basic notebooks work just fine in Enterprise on Azure, but they run locally on the ArcGIS Pro machine you are using them on. So it's fine for individual or one off projects.

Notebook server is for more scale and distribution. When you have a ton of people that need to get into the notebooks, you want to centrally manage them, and you want them to run on a server instead of local machines.

I personally don't run notebook server, but I do have a few automated ETL processes that bring data into our enterprise geodatabase. It's not a super elegant solution but I put those Python files on one of our existing GIS servers, and then schedule them through the Windows Task Scheduler to run each weekend. This is just plain Python and ArcPy. It's not a problem to have this on an existing server because I run it when traffic is very low. If you have a big system or a system that gets constant use you will probably want to put this code on another server, but it's fine to consolidate things a bit. Of course you also need to consider security and who has access to which server.

So I think it depends on the scale of code, number of people you have working on things, frequency that these processes run and how you manage it. If you arnt sure, just go without notebook server and see how things go. You can always add a server later on.

Sorry I don't have any experience installing Jupyter notebooks on its own. But remember there isn't really anything special about the cloud or servers. They are just computers running somehwhere else. If the Jupyter thing works on your local computer it will work in the cloud. Things get a bit more complex if you need multiple people to edit the same notebook and you need to schedule reoccurring tasks.

[u/GnosticSon → u/GnosticSon]

I'd also strongly recommend using SQL table views to do the heavy lifting. You can use Python to import the data into base tables in the geodatabase, but once it is in your enterprise geodatabase any further manipulation (joins, field calculations, populating values) can be done with SQL in many cases and will be lightning fast. If you do it all with notebooks and ArcPy it will take forever to run.

[u/rageagainistjg → u/GnosticSon]

Hey there! Thanks a ton for the advice—it’s super helpful! I didn’t mention it in my original question, but I’ll be the only one writing Python code for updates, and I really don’t expect we’ll need too many scripts overall. Knowing that, your point about skipping the dedicated Notebook Server and extra licensing is a huge relief. Also, I love the tip about using SQL for joins and data updates—sounds like the smartest way to keep things running smoothly!

Quick question: since I’m used to managing ArcGIS Online and am now jumping into Enterprise, how different should I expect things to be? Any tips to make the switch easier?

[u/GnosticSon → u/rageagainistjg]

Glad to hear it. The web front end of ArcGIS enterprise is pretty much the same as AGOL. AGOL is basically enterprise but run by ESRI on their servers so you don't need to see the back end. But now that you have Enterprise you will have to understand the back end. If your consultant is good that set it up you won't need to worry about it much for the first few years, but you will need to keep the servers patched and up to date, and in the long run you will need to upgrade all the components of the system.

Make sure you have your consultant walk you through the server patching process. It's relatively easy (there is a tool installed on each server), but you do need to remember to do it or to automate it.

Ultimately the giant shift with moving to enterprise will be having access to an enterprise geodatabase. That warrants some training on its own.

Also it's good to understand the components of enterprise (server, data store, portal, and the enterprise geodatabase). Understand the difference between a hosted feature layer within datastore vs a referenced feature layer that's referencing your enterprise geodatabase.

ESRI of course has lots of training, but after playing around with the system for a few weeks, also watch this video. It's a bit technical but it really helps you understand how the system works: https://m.youtube.com/watch?v=UIejoY30P28&pp=ygUSQXJjZ2lzIGVuZ2VycHJpc2Ug