If you are a quantitative analyst, Jupyter Notebook is a great application for researching signals, evaluating pricing models, and experimenting with data sets too large or complicated to do on spreadsheets. Jupyter Notebook makes it easy to run research, visualize results quickly and iteratively, and share results with others—but it lacks some important capabilities.
We’ve combined the flexibility and ease of use of Jupyter Notebook with the enterprise-grade scale and controls offered by Beacon Platform, to accelerate the path from research to production and time to value.
Four important features that we’d like to highlight are:
- Workspace management
- Version control
- Job scheduling and elastic compute
- Environment management
Figure 1: Uplifted Notebook Experience integrated with Beacon Platform
Workspace management
Notebook research often contains sensitive information, restricted intellectual property, or explorations that are not ready for general consumption. So it is important to be able to control access to keep these details within the appropriate team or group. Beacon Notebook leverages Beacon’s authentication and authorization capabilities to ensure that unauthorized users cannot read or write to protected areas in the user’s workspace. Other users see only the project folders that have been shared with them, based on their identity and roles.
Setting up a shared project is as simple as right-clicking on a file or folder of choice then using Beacon Notebook’s “Access Controls” menu to access and edit entitlements.
Entitlements on Beacon Notebook are managed via user, role, and group. Beacon Notebook is integrated with the rest of Beacon Platform, delivering automated updates to notebooks with new data, and restricting notebook access to specific batch jobs.
Figure 2: Role and group-based access control
Version control
Since notebooks are typically used in the context of research and experimentation, another important capability to have is version control. With Beacon Notebook, users have access to a simple, calendar-style interface to review earlier versions and see what has changed over time, without the complexity of a full software version control system such as Git. Notebook owners can delete past versions that are no longer needed, roll back changes to a previous version if an experiment doesn’t work out, or designate a specific version as the production or approved one. This last feature is great for continuing to experiment in a notebook without impacting other users or processes that use the production version.
Figure 3: Notebook versioning and tagging
Job scheduling and elastic compute
Notebook-based calculations can be complex and run across a large dataset or broad range of scenarios. With a normal Jupyter Notebook installation, the process usually runs locally, restricted by whatever processing power is available on the host the user was assigned to. With Beacon Notebook, users can continue to use this option for most of their work, but they can also leverage Beacon Cloud’s elastic compute infrastructure. By simply selecting the desired host type from Beacon Notebook’s Launcher menu, the notebook can be executed in one of Beacon’s cloud compute pools that dynamically spin up and down resources without any user involvement. These resources are readily extensible by each client, can be set up to support other languages, such as R, and can be configured to run in a multi-architecture setup, including leveraging GPUs for calculations.
Figure 4: Running notebooks and consoles on different compute pools
Another useful feature of Beacon Notebook is the ability to schedule notebooks to run at regular designated intervals. A simple calendar-like user interface allows users to select when they want the notebook to run, such as daily or weekly, and on what days and times. Beacon’s batch job scheduler then takes over, executes the notebook at the desired time, and writes the results back to the notebook once the job completes. With Beacon’s version control, users can easily select which results they want to see from a simple dated list of the document’s history. Users also have the ability to generate email reports from their scheduled notebooks, making it easy to monitor and share results.
Environment management
Users often need to configure their working environment with specific libraries or packages, which may include hardware drivers (in the case of GPU hosts), language bindings, or various development packages. Beacon Notebook simplifies this process by offering a convenient way to customize the environment for each notebook. With Beacon Notebook, users can tailor the environment settings for individual notebooks.
There are two ways to do this:
- Package Selection – Users can choose a set of Pip and Conda packages they want to include in their notebook’s environment.
- Custom Environment Creation – Users can directly create their desired environment by running commands from within the notebook itself and then save a snapshot of these changes.
Once the notebook environment is set up, other users and processes can easily recreate the same environment, ensuring that the notebook functions as the author intended.
Figure 5: Notebook running environment management
Accelerate the path from idea to deployment
Beacon Notebook makes the powerful financial and risk analytics available in Beacon Platform accessible to a wider audience of users. Data scientists, quant analysts, and citizen developers can use notebooks directly, without having to rely on IT or core developers. With access to Beacon’s analytics, data warehouse tools, and computing power, our clients are finding innovative ways to model and manage risk across all asset classes, build differentiating strategies, and generate better insights, faster. Once they have built, refined, and verified their model or analysis, the same code can be moved into Beacon’s version control system and begin its journey to deployment, without having to rewrite and retest in a different language and environment, accelerating the path from idea to production.
