cowbird.handlers.impl.filesystem

Attributes

HANDLER_WORKSPACE_DIR_PARAM

JSON

SettingsType

LOGGER

DEFAULT_NOTEBOOKS_DIR_NAME

DEFAULT_PUBLIC_WORKSPACE_WPS_OUTPUTS_SUBPATH

DEFAULT_WPS_OUTPUTS_RES_NAME

DEFAULT_SECURE_DATA_PROXY_NAME

DEFAULT_USER_WPS_OUTPUTS_DIR_NAME

Classes

HandlerFactory

Create handler instance using handler name.

Handler

Helper class that provides a standard way to create an ABC using

FSMonitor

Interface being called when something changes on the filesystem.

Monitoring

Class handling file system monitoring and registering listeners.

Permission

Define every property required to set a permission in Magpie.

FileSystem

Keep the proper directory structure in sync with the platform.

Functions

apply_new_path_permissions(→ None)

Applies new permissions to a path, if required.

get_logger(→ logging.Logger)

Immediately sets the logger level to avoid duplicate log outputs from the root logger and this logger when

Module Contents

class cowbird.handlers.impl.filesystem.HandlerFactory[source]

Create handler instance using handler name.

create_handler(name: Literal['Catalog']) cowbird.handlers.impl.catalog.Catalog[source]
create_handler(name: Literal['FileSystem']) cowbird.handlers.impl.filesystem.FileSystem
create_handler(name: Literal['Geoserver']) cowbird.handlers.impl.geoserver.Geoserver
create_handler(name: Literal['Magpie']) cowbird.handlers.impl.magpie.Magpie
create_handler(name: Literal['Nginx']) cowbird.handlers.impl.nginx.Nginx
create_handler(name: Literal['Thredds']) cowbird.handlers.impl.thredds.Thredds
create_handler(name: str) cowbird.handlers.handler.Handler | None

Instantiates a new Handler implementation using its name, overwriting an existing instance if required.

get_handler(name: Literal['Catalog']) cowbird.handlers.impl.catalog.Catalog[source]
get_handler(name: Literal['FileSystem']) cowbird.handlers.impl.filesystem.FileSystem
get_handler(name: Literal['Geoserver']) cowbird.handlers.impl.geoserver.Geoserver
get_handler(name: Literal['Magpie']) cowbird.handlers.impl.magpie.Magpie
get_handler(name: Literal['Nginx']) cowbird.handlers.impl.nginx.Nginx
get_handler(name: Literal['Thredds']) cowbird.handlers.impl.thredds.Thredds
get_handler(name: str) cowbird.handlers.handler.Handler | None

Instantiates a Handler implementation using its name if it doesn’t exist or else returns the existing one from cache.

get_active_handlers() List[cowbird.handlers.handler.Handler][source]

Return a sorted list by priority of Handler implementation activated in the config.

cowbird.handlers.impl.filesystem.HANDLER_WORKSPACE_DIR_PARAM: AnyHandlerParameter = 'workspace_dir'[source]
class cowbird.handlers.impl.filesystem.Handler(settings: cowbird.typedefs.SettingsType, name: str, **kwargs: Any)[source]

Bases: abc.ABC

Helper class that provides a standard way to create an ABC using inheritance.

Parameters:
  • settings – Cowbird settings for convenience

  • name – Handler name

  • kwargs – The base class handle, but doesn’t require the following variables:

  • url – Location of the web service represented by the cowbird handler

  • workspace_dir – Workspace directory

  • priority – Relative priority between handlers while handling events. Lower value has higher priority, default value is last.

__slots__

Handler interface used to notify implemented handlers of users/permissions changes.

Todo

At some point we will need a consistency function that goes through all Magpie users and make sure that handlers are up-to-date.

property required_params: List[AnyHandlerParameter]
Abstractmethod:

json() cowbird.typedefs.JSON[source]
_user_workspace_dir(user_name: str) str[source]
abstract user_created(user_name: str) None[source]
abstract user_deleted(user_name: str) None[source]
abstract permission_created(permission: cowbird.permissions_synchronizer.Permission) None[source]
abstract permission_deleted(permission: cowbird.permissions_synchronizer.Permission) None[source]
abstract resync() None[source]
class cowbird.handlers.impl.filesystem.FSMonitor[source]

Bases: abc.ABC

Interface being called when something changes on the filesystem.

static get_instance() FSMonitor | None[source]
Abstractmethod:

Must return an instance of the class implementing FSMonitor.

abstract on_created(path: str) None[source]

Called when a new path is found.

Parameters:

path – Absolute path of a new file/directory

abstract on_deleted(path: str) None[source]

Called when a path is deleted.

Parameters:

path – Absolute path of a new file/directory

abstract on_modified(path: str) None[source]

Called when a path is updated.

Parameters:

path – Absolute path of a new file/directory

class cowbird.handlers.impl.filesystem.Monitoring(config: cowbird.typedefs.AnySettingsContainer = None)[source]

Class handling file system monitoring and registering listeners.

Todo

At some point we will need a consistency function that goes through all monitored folder and make sure that monitoring handlers are up to date.

Initialize the monitoring instance from configured application settings.

Parameters:

config – AnySettingsContainer object from which the db can be retrieved. The default value of None is only there to disable pylint E1120. The singleton instance must be initialized with a proper config and after that the init function should not be hit.

start() None[source]

Load existing monitors and start the monitoring.

register(path: str, recursive: bool, cb_monitor: cowbird.monitoring.fsmonitor.FSMonitor | Type[cowbird.monitoring.fsmonitor.FSMonitor] | str) cowbird.monitoring.monitor.Monitor | None[source]

Register a monitor for a specific path and start it. If a monitor already exists for the specific path/cb_monitor combination it is directly returned. If this monitor was not recursively monitoring its path and the recursive flag is now true, this one take precedence and the monitor is updated accordingly. If the recursive flag was true, and now it is false it has no effect.

Parameters:
  • path – Path to monitor

  • recursive – Monitor subdirectory recursively?

  • cb_monitor – FSMonitor for which an instance is created and events are sent Can be an object, a class type implementing FSMonitor or a string containing module and class name.

Returns:

The monitor registered or already existing for the specific path/cb_monitor combination. Note that the monitor is not created/returned if a MonitorException occurs.

unregister(path: str, cb_monitor: cowbird.monitoring.fsmonitor.FSMonitor | Type[cowbird.monitoring.fsmonitor.FSMonitor] | str) bool[source]

Stop a monitor and unregister it.

Parameters:
  • path – Path used by the monitor

  • cb_monitor – FSMonitor object to remove Can be an object, a class type implementing FSMonitor or a string containing module and class name.

Returns:

True if the monitor is found and successfully stopped, False otherwise

unregister_all() None[source]

Stops and unregisters all monitors.

class cowbird.handlers.impl.filesystem.Permission(service_name: str, service_type: str, resource_id: int, resource_full_name: str, name: str, access: str, scope: str, user: str = None, group: str = None, resource_display_name: str = None)[source]

Define every property required to set a permission in Magpie.

__eq__(other: Permission) bool[source]

Return self==value.

get_full_permission_value() str[source]

Returns the full permission value, consisting of the name-access-scope values.

cowbird.handlers.impl.filesystem.JSON[source]
cowbird.handlers.impl.filesystem.SettingsType[source]
cowbird.handlers.impl.filesystem.apply_new_path_permissions(path: str, is_readable: bool, is_writable: bool, is_executable: bool) None[source]

Applies new permissions to a path, if required.

cowbird.handlers.impl.filesystem.get_logger(name: str, level: int | None = None, force_stdout: bool = None, message_format: str | None = None, datetime_format: str | None = None) logging.Logger[source]

Immediately sets the logger level to avoid duplicate log outputs from the root logger and this logger when level is logging.NOTSET.

cowbird.handlers.impl.filesystem.LOGGER[source]
cowbird.handlers.impl.filesystem.DEFAULT_NOTEBOOKS_DIR_NAME = 'notebooks'[source]
cowbird.handlers.impl.filesystem.DEFAULT_PUBLIC_WORKSPACE_WPS_OUTPUTS_SUBPATH = 'public/wps_outputs'[source]
cowbird.handlers.impl.filesystem.DEFAULT_WPS_OUTPUTS_RES_NAME = 'wps_outputs'[source]
cowbird.handlers.impl.filesystem.DEFAULT_SECURE_DATA_PROXY_NAME = 'secure-data-proxy'[source]
cowbird.handlers.impl.filesystem.DEFAULT_USER_WPS_OUTPUTS_DIR_NAME = 'wps_outputs'[source]
class cowbird.handlers.impl.filesystem.FileSystem(settings: cowbird.typedefs.SettingsType, name: str, jupyterhub_user_data_dir: str, wps_outputs_dir: str, secure_data_proxy_name: str = DEFAULT_SECURE_DATA_PROXY_NAME, wps_outputs_res_name: str = DEFAULT_WPS_OUTPUTS_RES_NAME, notebooks_dir_name: str = DEFAULT_NOTEBOOKS_DIR_NAME, public_workspace_wps_outputs_subpath: str = DEFAULT_PUBLIC_WORKSPACE_WPS_OUTPUTS_SUBPATH, user_wps_outputs_dir_name: str = DEFAULT_USER_WPS_OUTPUTS_DIR_NAME, **kwargs: Any)[source]

Bases: cowbird.handlers.handler.Handler, cowbird.monitoring.fsmonitor.FSMonitor

Keep the proper directory structure in sync with the platform.

Create the file system instance.

Parameters:
  • settings – Cowbird settings for convenience

  • name – Handler name

  • jupyterhub_user_data_dir – Path to the JupyterHub user data directory, which will be symlinked to the working directory

  • wps_outputs_dir – Path to the wps outputs directory

  • secure_data_proxy_name – Name of the secure-data-proxy service found on Magpie

  • wps_outputs_res_name – Name of the WPS outputs resource found on Magpie under the secure-data-proxy service

  • notebooks_dir_name – Name of the symlink directory found in the user workspace and which directs to the user’s notebook directory

  • public_workspace_wps_outputs_subpath – Subpath to the directory containing hardlinks to the public WPS outputs data

  • user_wps_outputs_dir_name – Name of the directory found in the user workspace and which contains the hardlinks to the user WPS outputs data

required_params[source]
start_wps_outputs_monitoring(monitoring: cowbird.monitoring.monitoring.Monitoring) None[source]
get_user_workspace_dir(user_name: str) str[source]
get_user_workspace_wps_outputs_dir(user_name: str) str[source]
get_public_workspace_wps_outputs_dir() str[source]
_get_jupyterhub_user_data_dir(user_name: str) str[source]
user_created(user_name: str) None[source]
user_deleted(user_name: str) None[source]
static get_instance() FileSystem[source]

Return the FileSystem singleton instance from the class name used to retrieve the FSMonitor from the DB.

_get_secure_data_proxy_file_perms(src_path: str, user_name: str) Tuple[bool, bool][source]

Finds a route from the secure-data-proxy service that matches the resource path (or one of its parent resource) and gets the user permissions on that route.

update_secure_data_proxy_path_perms(src_path: str, user_name: str) bool[source]

Gets a path’s permissions from the secure-data-proxy service and updates the file system permissions accordingly.

Returns a boolean to indicate if the user should have some type of access to the path or not.

Creates a hardlink path from a source file, if the user has access rights.

on_created(path: str) None[source]

Call when a new path is found.

Parameters:

path – Absolute path of a new file/directory

on_modified(path: str) None[source]

Called when a path is updated.

Parameters:

path – Absolute path of a new file/directory

Deletes the hardlink path that corresponds to the input source path.

Returns a bool to indicate if a hardlink path was deleted or not.

on_deleted(path: str) None[source]

Called when a path is deleted.

Parameters:

path – Absolute path of a new file/directory

_check_if_res_from_secure_data_proxy(res_tree: List[cowbird.typedefs.JSON]) bool[source]

Checks if the resource is part of a secure-data-proxy service of type API.

_update_permissions_on_filesystem(permission: cowbird.permissions_synchronizer.Permission) None[source]
permission_created(permission: cowbird.permissions_synchronizer.Permission) None[source]
permission_deleted(permission: cowbird.permissions_synchronizer.Permission) None[source]
resync() None[source]

Resync operation, regenerating required links (user_workspace, wps_outputs, …)