{ "cells": [ { "cell_type": "markdown", "id": "ab6a4ad2", "metadata": {}, "source": [ "# Package overview tour" ] }, { "cell_type": "markdown", "id": "d1278d27", "metadata": {}, "source": [ "Requirement on the shell:\n", "* Need `cat` command. If you use Windows command prompts, replace `cat` by `type` at each IPython magic line starting with `!`.\n", "\n", "Requirement on python packages:\n", "* pandas: `pip install pandas`" ] }, { "cell_type": "markdown", "id": "ed0f8fc5", "metadata": {}, "source": [ "This tutorial describes what our package can do and how to use it. This does not cover all functions or use cases but shows the most interesting features." ] }, { "cell_type": "markdown", "id": "9d4d5347", "metadata": {}, "source": [ "Let's load the core I/O interface class: eXtended-I/O abbreviated as X-I/O or XIO" ] }, { "cell_type": "code", "execution_count": 1, "id": "95a94282", "metadata": { "scrolled": true }, "outputs": [], "source": [ "from brane import ExtendedIO as xio" ] }, { "cell_type": "markdown", "id": "ae44397c", "metadata": {}, "source": [ "## Read / Write operations via eXtended-I/O" ] }, { "cell_type": "markdown", "id": "2f082049", "metadata": {}, "source": [ "### text" ] }, { "cell_type": "markdown", "id": "55db96d8", "metadata": {}, "source": [ "First of all, let's save the following text as a normal text file with XIO:" ] }, { "cell_type": "code", "execution_count": 2, "id": "810b5c53", "metadata": {}, "outputs": [], "source": [ "text = \"\"\"\n", "name,role,birthyear\n", "Alice,sender,1978\n", "Bob,receiver,1978\n", "Carol,,1984\n", "Eve,eavesdropper,1988\n", "Mallory,attacker,2003\n", "Walter,warden,\n", "Ivan,issuer,2002\n", "\"\"\".strip()" ] }, { "cell_type": "markdown", "id": "f088efdc", "metadata": {}, "source": [ "It's very simple. Specify the text and the path to write." ] }, { "cell_type": "code", "execution_count": 3, "id": "c5955ba8", "metadata": {}, "outputs": [], "source": [ "xio.write(text, \"text.txt\")" ] }, { "cell_type": "markdown", "id": "63daf9e0", "metadata": {}, "source": [ "And you can see that this is written as just an ordinary text." ] }, { "cell_type": "code", "execution_count": 4, "id": "d2d7a0b8", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "name,role,birthyear\r\n", "Alice,sender,1978\r\n", "Bob,receiver,1978\r\n", "Carol,,1984\r\n", "Eve,eavesdropper,1988\r\n", "Mallory,attacker,2003\r\n", "Walter,warden,\r\n", "Ivan,issuer,2002" ] } ], "source": [ "!cat ./text.txt # this line works for Linux" ] }, { "cell_type": "markdown", "id": "7e627aab", "metadata": {}, "source": [ "The reading the saved file is also simple:" ] }, { "cell_type": "code", "execution_count": 5, "id": "3a104cf5", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "name,role,birthyear\n", "Alice,sender,1978\n", "Bob,receiver,1978\n", "Carol,,1984\n", "Eve,eavesdropper,1988\n", "Mallory,attacker,2003\n", "Walter,warden,\n", "Ivan,issuer,2002\n" ] } ], "source": [ "text_reload = xio.read(\"text.txt\")\n", "assert type(text_reload) == str\n", "print(text_reload)" ] }, { "cell_type": "markdown", "id": "1924e45a", "metadata": {}, "source": [ "The reloaded one is of string type and same as the orginal one." ] }, { "cell_type": "code", "execution_count": 6, "id": "f795b847", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "True" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "text == text_reload" ] }, { "cell_type": "markdown", "id": "0523c477", "metadata": {}, "source": [ "Next, save this text as a csv file." ] }, { "cell_type": "code", "execution_count": 7, "id": "dd341567", "metadata": {}, "outputs": [], "source": [ "xio.write(text, \"actor.csv\")" ] }, { "cell_type": "markdown", "id": "6c57ce4f", "metadata": {}, "source": [ "Notice the extension is just the symbol of the format but meaningless to the filesystem.\n", "In this case, the content of this file is still text itself." ] }, { "cell_type": "code", "execution_count": 8, "id": "076e5794", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "name,role,birthyear\r\n", "Alice,sender,1978\r\n", "Bob,receiver,1978\r\n", "Carol,,1984\r\n", "Eve,eavesdropper,1988\r\n", "Mallory,attacker,2003\r\n", "Walter,warden,\r\n", "Ivan,issuer,2002" ] } ], "source": [ "!cat ./actor.csv # this line works for Linux" ] }, { "cell_type": "markdown", "id": "3bcbe79c", "metadata": {}, "source": [ "### csv" ] }, { "cell_type": "markdown", "id": "cc517bda", "metadata": {}, "source": [ "Now try reading the csv file. It's just a text with a csv extension in the filename." ] }, { "cell_type": "code", "execution_count": 9, "id": "92c2de1c", "metadata": { "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n" ] }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
namerolebirthyear
0Alicesender1978.0
1Bobreceiver1978.0
2CarolNaN1984.0
3Eveeavesdropper1988.0
4Malloryattacker2003.0
5WalterwardenNaN
6Ivanissuer2002.0
\n", "
" ], "text/plain": [ " name role birthyear\n", "0 Alice sender 1978.0\n", "1 Bob receiver 1978.0\n", "2 Carol NaN 1984.0\n", "3 Eve eavesdropper 1988.0\n", "4 Mallory attacker 2003.0\n", "5 Walter warden NaN\n", "6 Ivan issuer 2002.0" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df = xio.read(\"actor.csv\")\n", "print(type(df))\n", "df" ] }, { "cell_type": "markdown", "id": "221b2fca", "metadata": {}, "source": [ "In this turn, we find that the loaded object is not a text but pandas DataFrame.\n", "If you would like to use the builtin's csv package in some reason, you can do it by passing the module_name as 'csv'." ] }, { "cell_type": "code", "execution_count": 10, "id": "ba503cc3", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n" ] }, { "data": { "text/plain": [ "[['name', 'role', 'birthyear'],\n", " ['Alice', 'sender', '1978'],\n", " ['Bob', 'receiver', '1978'],\n", " ['Carol', '', '1984'],\n", " ['Eve', 'eavesdropper', '1988'],\n", " ['Mallory', 'attacker', '2003'],\n", " ['Walter', 'warden', ''],\n", " ['Ivan', 'issuer', '2002']]" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df = xio.read(\"actor.csv\", module_name=\"csv\")\n", "print(type(df))\n", "df" ] }, { "cell_type": "markdown", "id": "6009a145", "metadata": {}, "source": [ "Again, you can specify the path argument and the pandas module for module_name to ensure. It's better to add type annotation if you know the type of the loaded object." ] }, { "cell_type": "code", "execution_count": 11, "id": "bf29fb28", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n" ] }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
namerolebirthyear
0Alicesender1978.0
1Bobreceiver1978.0
2CarolNaN1984.0
3Eveeavesdropper1988.0
4Malloryattacker2003.0
5WalterwardenNaN
6Ivanissuer2002.0
\n", "
" ], "text/plain": [ " name role birthyear\n", "0 Alice sender 1978.0\n", "1 Bob receiver 1978.0\n", "2 Carol NaN 1984.0\n", "3 Eve eavesdropper 1988.0\n", "4 Mallory attacker 2003.0\n", "5 Walter warden NaN\n", "6 Ivan issuer 2002.0" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import pandas as pd\n", "df: pd.DataFrame = xio.read(path=\"actor.csv\", module_name=\"pandas\")\n", "print(type(df))\n", "df" ] }, { "cell_type": "markdown", "id": "a9e23a54", "metadata": {}, "source": [ "Let's save the pandas dataframe. As you may know, this will save the index too:" ] }, { "cell_type": "code", "execution_count": 12, "id": "eaf4ce01", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "name,role,birthyear\r\n", "Alice,sender,1978.0\r\n", "Bob,receiver,1978.0\r\n", "Carol,,1984.0\r\n", "Eve,eavesdropper,1988.0\r\n", "Mallory,attacker,2003.0\r\n", "Walter,warden,\r\n", "Ivan,issuer,2002.0\r\n" ] } ], "source": [ "xio.write(df, \"actor_w_index.csv\")\n", "!cat ./actor_w_index.csv # this line works for Linux" ] }, { "cell_type": "markdown", "id": "7cf3791e", "metadata": {}, "source": [ "Wow, no index appears. This is because some most common parameters are set in advance.\n", "For pandas, `index=None` is such a parameter." ] }, { "cell_type": "markdown", "id": "831d1b9e", "metadata": {}, "source": [ "Of course, you can specify the index too." ] }, { "cell_type": "code", "execution_count": 13, "id": "859679d3", "metadata": { "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "name,role,birthyear\r\n", "Alice,sender,1978.0\r\n", "Bob,receiver,1978.0\r\n", "Carol,,1984.0\r\n", "Eve,eavesdropper,1988.0\r\n", "Mallory,attacker,2003.0\r\n", "Walter,warden,\r\n", "Ivan,issuer,2002.0\r\n" ] } ], "source": [ "xio.write(obj=df, path=\"actor_w_name_index.csv\", index=True)\n", "!cat ./actor_w_name_index.csv # this line works for Linux" ] }, { "cell_type": "markdown", "id": "792c8393", "metadata": {}, "source": [ "Here, for clarity, we put the keyword arguments like obj and path," ] }, { "cell_type": "markdown", "id": "f807db93", "metadata": {}, "source": [ "## Customization" ] }, { "cell_type": "markdown", "id": "d4f1bcf5", "metadata": {}, "source": [ "In yhis section, we\n", "\n", "* treat the Python dictionary but define its wrapper class\n", "* consider new own format or extension .hello" ] }, { "cell_type": "markdown", "id": "bb82ddbc", "metadata": {}, "source": [ "### Define own module" ] }, { "cell_type": "markdown", "id": "7252556c", "metadata": {}, "source": [ "For that purpose, let's define our own format 'hello' with its class' implementation as follows:\n", "\n", "* header (1st line): a symbol which plays role as a separator in the following lines\n", "* body (any other sequent lines): each line consisting of key and value with the separator specified in the header." ] }, { "cell_type": "code", "execution_count": 14, "id": "b3ac9183", "metadata": {}, "outputs": [], "source": [ "from __future__ import annotations\n", "from typing import Union\n", "\n", "class HelloClass():\n", " # hello object which wraps a Python dictionary\n", " def __init__(self, mapper: dict[str, Union[str, int]]):\n", " self.mapper = mapper\n", "\n", " def __repr__(self) -> str:\n", " return self.mapper.__repr__()\n", "\n", "class HelloIO():\n", " # hello format IO module\n", " @staticmethod\n", " def load(path: str) -> HelloClass:\n", " with open(path, \"r\") as f:\n", " file: str = f.read()\n", " sep, *lines = file.split(\"\\n\")\n", " data: dict = {}\n", " for l in lines:\n", " k, v = l.split(sep)\n", " data[k] = v\n", " return HelloClass(mapper=data)\n", "\n", " @staticmethod\n", " def dump(mapper: HelloClass, path: str, sep: str = \": \"):\n", " with open(path, \"w\") as f:\n", " f.write(f\"{sep}\")\n", " for key, value in obj.mapper.items():\n", " f.write(f\"\\n{key}{sep}{value}\")" ] }, { "cell_type": "markdown", "id": "05c49b01", "metadata": {}, "source": [ "Now, test it:" ] }, { "cell_type": "code", "execution_count": 15, "id": "a20e9341", "metadata": {}, "outputs": [], "source": [ "obj = HelloClass(mapper={\n", " \"Jan\": 1,\n", " \"Feb\": 2,\n", " \"Mar\": 3,\n", "})" ] }, { "cell_type": "markdown", "id": "ca263c21", "metadata": {}, "source": [ "Save it." ] }, { "cell_type": "code", "execution_count": 16, "id": "5b6cf98d", "metadata": {}, "outputs": [], "source": [ "HelloIO.dump(mapper=obj, path=\"test.hello\")" ] }, { "cell_type": "code", "execution_count": 17, "id": "cb22ccf4", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ ": \r\n", "Jan: 1\r\n", "Feb: 2\r\n", "Mar: 3" ] } ], "source": [ "!cat ./test.hello" ] }, { "cell_type": "markdown", "id": "328b4fc5", "metadata": {}, "source": [ "Load it." ] }, { "cell_type": "code", "execution_count": 18, "id": "7d93a225", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'Jan': '1', 'Feb': '2', 'Mar': '3'}" ] }, "execution_count": 18, "metadata": {}, "output_type": "execute_result" } ], "source": [ "HelloIO.load(path=\"test.hello\")" ] }, { "cell_type": "markdown", "id": "4c7937d6", "metadata": {}, "source": [ "Good." ] }, { "cell_type": "markdown", "id": "c2559c94", "metadata": {}, "source": [ "### Define new Module subclass" ] }, { "cell_type": "markdown", "id": "5eb2f553", "metadata": {}, "source": [ "Next task is to register our hello format to X-I/O. For that purpose, let's define a new class inheritating the Module class.\n", "This class defines the common interface i.e. it has read & write method." ] }, { "cell_type": "code", "execution_count": 19, "id": "5ad3c463", "metadata": {}, "outputs": [], "source": [ "from brane.core import Module\n", "\n", "class HelloModule(Module):\n", " name = \"hello\" # ID of this Module class\n", "\n", " @classmethod\n", " def read(cls, path: str, *args, **kwargs):\n", " return HelloIO.load(path=path)\n", "\n", " @classmethod\n", " def write(cls, obj, path, *args, **kwargs):\n", " return HelloIO.dump(mapper=obj, path=path)" ] }, { "cell_type": "markdown", "id": "93ba39a9", "metadata": {}, "source": [ "You must define three attributes at this class:\n", "\n", "* name (propetry): This is the ID of the new module.\n", "* read (classmethod): This defines the reading/loading process with the keyword arguments path at least.\n", "* write (classmethod): This defines the writing/saving process with the keyword arguments obj & path at least." ] }, { "cell_type": "markdown", "id": "6b04bd0e", "metadata": {}, "source": [ "Test it again." ] }, { "cell_type": "code", "execution_count": 20, "id": "95b72e32", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'Jan': '1', 'Feb': '2', 'Mar': '3'}" ] }, "execution_count": 20, "metadata": {}, "output_type": "execute_result" } ], "source": [ "HelloModule.read(obj=obj, path=\"test.hello\")" ] }, { "cell_type": "code", "execution_count": 21, "id": "a07807ff", "metadata": {}, "outputs": [], "source": [ "HelloModule.write(obj=obj, path=\"test2.hello\")" ] }, { "cell_type": "code", "execution_count": 22, "id": "b85cb1be", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ ": \r\n", "Jan: 1\r\n", "Feb: 2\r\n", "Mar: 3" ] } ], "source": [ "!cat ./test2.hello" ] }, { "cell_type": "markdown", "id": "616b733d", "metadata": {}, "source": [ "No problem at all." ] }, { "cell_type": "markdown", "id": "c1d65143", "metadata": {}, "source": [ "### Define new Format subclass" ] }, { "cell_type": "markdown", "id": "f0b74cf8", "metadata": {}, "source": [ "At this stage, there is no connection between HelloModule and the hello extension shown in paths.\n", "Then, we must implement another class called `Format` which connect the above reading/writing module and the extension." ] }, { "cell_type": "code", "execution_count": 23, "id": "df9f93a1", "metadata": {}, "outputs": [], "source": [ "from brane.core import Format\n", "\n", "class HelloFormat(Format):\n", " name = \"hello\" # ID of this Format class\n", " module = HelloModule\n", " default_extension = \"hello\" # the extension in the path" ] }, { "cell_type": "markdown", "id": "e0ecf25d", "metadata": {}, "source": [ "* name (propetry): This is the ID of the new format.\n", "* module (property): Assign the correspnding `Module` subclass.\n", "* default_extension (property): The extension name." ] }, { "cell_type": "code", "execution_count": 24, "id": "787b14a9", "metadata": { "scrolled": true }, "outputs": [ { "data": { "text/plain": [ "{'Jan': '1', 'Feb': '2', 'Mar': '3'}" ] }, "execution_count": 24, "metadata": {}, "output_type": "execute_result" } ], "source": [ "xio.read(\"test.hello\")" ] }, { "cell_type": "markdown", "id": "5c9f9e6b", "metadata": {}, "source": [ "OK, great work ! Now, the brane I/O choose the correct module based on the extension." ] }, { "cell_type": "markdown", "id": "6371a92c", "metadata": {}, "source": [ "### Define new Object subclass" ] }, { "cell_type": "markdown", "id": "f7507a20", "metadata": {}, "source": [ "Finally, let's save our Hello object in our Hello format i.e. with the `.hello` extension." ] }, { "cell_type": "code", "execution_count": 25, "id": "4ba15ab1", "metadata": {}, "outputs": [], "source": [ "from brane.core import Object\n", "\n", "class HelloObject(Object):\n", " format = HelloFormat\n", " module = HelloModule\n", " object_type = HelloClass" ] }, { "cell_type": "markdown", "id": "81c04f71", "metadata": {}, "source": [ "In our case, it's still simple:\n", "\n", "* module (property): The corresponding `Module` subclass.\n", "* format (property): The corresponding `Format` subclass.\n", "* object_type (property): The type of the target objects, here, `HelloClass`." ] }, { "cell_type": "code", "execution_count": 26, "id": "9851c6ad", "metadata": {}, "outputs": [], "source": [ "xio.write(obj=obj, path=\"auto.hello\")" ] }, { "cell_type": "code", "execution_count": 27, "id": "3916ff1b", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ ": \r\n", "Jan: 1\r\n", "Feb: 2\r\n", "Mar: 3" ] } ], "source": [ "!cat ./auto.hello # this line works for Linux" ] }, { "cell_type": "markdown", "id": "c7aa0bfe", "metadata": {}, "source": [ "Now, we've learned the basic definition & registration of our own I/O to the eXtend-I/O." ] }, { "cell_type": "markdown", "id": "13dfc396", "metadata": {}, "source": [ "## Hook system" ] }, { "cell_type": "markdown", "id": "9c63079e", "metadata": {}, "source": [ "### Check the pre-existing hooks" ] }, { "cell_type": "markdown", "id": "ea82f529", "metadata": {}, "source": [ "Let's save the following text to a new directory named `quotes`:" ] }, { "cell_type": "code", "execution_count": 28, "id": "991af930", "metadata": {}, "outputs": [], "source": [ "text = \"\"\"\n", "Nobody ever figures out what life is all about, and it doesn't matter.\n", "Explore the world.\n", "Nearly everything is really interesting if you go into it deeply enough.\n", "\"\"\".strip()" ] }, { "cell_type": "markdown", "id": "6df8087c", "metadata": {}, "source": [ "First, try it without making the directory." ] }, { "cell_type": "code", "execution_count": 29, "id": "e2baaa5f", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[Errno 2] No such file or directory: 'quotes/most_like.txt'\n" ] } ], "source": [ "try:\n", " with open(\"quotes/most_like.txt\", \"w\") as f:\n", " f.write(text)\n", "except FileNotFoundError as e:\n", " print(e)" ] }, { "cell_type": "markdown", "id": "a80d6685", "metadata": {}, "source": [ "Of course, it fails as we expected. However," ] }, { "cell_type": "code", "execution_count": 30, "id": "ecdc1baf", "metadata": {}, "outputs": [], "source": [ "from brane.core import Object" ] }, { "cell_type": "code", "execution_count": 31, "id": "9c354480", "metadata": {}, "outputs": [], "source": [ "xio.write(obj=text, path=\"quotes/most_like.txt\")" ] }, { "cell_type": "code", "execution_count": 32, "id": "3ddde0ba", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Nobody ever figures out what life is all about, and it doesn't matter.\r\n", "Explore the world.\r\n", "Nearly everything is really interesting if you go into it deeply enough." ] } ], "source": [ "!cat quotes/most_like.txt # this line works for Linux" ] }, { "cell_type": "markdown", "id": "532ea02c", "metadata": {}, "source": [ "where no previous error happens if we save it through X-I/O ! The reason is that this called a hook function which creates the missing directory before saving. This can be checked by accessing the attribute `pre_write`:" ] }, { "cell_type": "code", "execution_count": 33, "id": "eb5e81da", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "1. 1269bcdeaab3db08: " ] }, "execution_count": 33, "metadata": {}, "output_type": "execute_result" } ], "source": [ "xio.pre_write" ] }, { "cell_type": "markdown", "id": "e172383c", "metadata": {}, "source": [ "where you see two strings at least except a number on top:\n", "* a hex sequence is a hook ID automatically assigned on the left\n", "* `` points to the python function named `create_parent_directory` which is exactly called at the above execution." ] }, { "cell_type": "markdown", "id": "1267faac", "metadata": {}, "source": [ "### Add new hook" ] }, { "cell_type": "markdown", "id": "205c956f", "metadata": {}, "source": [ "Let's consider new hooks:\n", "* remove the line break (`\\n`) on loading any text\n", "* put the copyright symbol on the last on saving any text" ] }, { "cell_type": "markdown", "id": "65f4b5ed", "metadata": {}, "source": [ "First of all, check the default behaviour:" ] }, { "cell_type": "code", "execution_count": 34, "id": "61f7ca2b", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "\"Nobody ever figures out what life is all about, and it doesn't matter.\\nExplore the world.\\nNearly everything is really interesting if you go into it deeply enough.\"" ] }, "execution_count": 34, "metadata": {}, "output_type": "execute_result" } ], "source": [ "xio.read(\"quotes/most_like.txt\")" ] }, { "cell_type": "markdown", "id": "bd20e092", "metadata": {}, "source": [ "OK. Now, define a function replacing any line break by a space." ] }, { "cell_type": "code", "execution_count": 35, "id": "c14dc884", "metadata": {}, "outputs": [], "source": [ "def remove_linebreaks(context):\n", " obj = context[\"object\"] # Currently, you must put this line for all hooks. Then, `obj` variable is the loaded object on loading.\n", " if isinstance(obj, str):\n", " return obj.replace(\"\\n\", \" \")" ] }, { "cell_type": "markdown", "id": "28037f30", "metadata": {}, "source": [ "And add it." ] }, { "cell_type": "code", "execution_count": 36, "id": "f1e80a40", "metadata": {}, "outputs": [], "source": [ "xio.register_post_read_hook(hook=remove_linebreaks)" ] }, { "cell_type": "markdown", "id": "3ade914d", "metadata": {}, "source": [ "Now, read it again" ] }, { "cell_type": "code", "execution_count": 37, "id": "a642fdf5", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "\"Nobody ever figures out what life is all about, and it doesn't matter. Explore the world. Nearly everything is really interesting if you go into it deeply enough.\"" ] }, "execution_count": 37, "metadata": {}, "output_type": "execute_result" } ], "source": [ "loaded_text: str = xio.read(\"quotes/most_like.txt\")\n", "loaded_text" ] }, { "cell_type": "markdown", "id": "e4d21a09", "metadata": {}, "source": [ "You see no line breaks now." ] }, { "cell_type": "markdown", "id": "2c3d2395", "metadata": {}, "source": [ "Next, let'a add copyright on the last line for every text." ] }, { "cell_type": "code", "execution_count": 38, "id": "e5cdf48e", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'© 2022 Richard P.Feynman'" ] }, "execution_count": 38, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import datetime\n", "\n", "author_name: str = \"Richard P.Feynman\"\n", "copyright: str = f\"© {datetime.datetime.today().year} {author_name}\"\n", "copyright" ] }, { "cell_type": "markdown", "id": "c4291cd4", "metadata": {}, "source": [ "In the same way as above, define the function and register it:" ] }, { "cell_type": "code", "execution_count": 39, "id": "0abaa38f", "metadata": {}, "outputs": [], "source": [ "def append_copyright(context):\n", " obj = context[\"object\"]\n", " if isinstance(obj, str):\n", " return obj + \"\\n\" + copyright" ] }, { "cell_type": "code", "execution_count": 40, "id": "581f4e18", "metadata": {}, "outputs": [], "source": [ "xio.register_pre_write_hook(hook=append_copyright)" ] }, { "cell_type": "markdown", "id": "73d6db0d", "metadata": {}, "source": [ "You see the two hooks are not registered at the pre-write hooks." ] }, { "cell_type": "code", "execution_count": 41, "id": "4c83db01", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "1. 1269bcdeaab3db08: \n", "2. 32e966b6991c51b9: " ] }, "execution_count": 41, "metadata": {}, "output_type": "execute_result" } ], "source": [ "xio.pre_write" ] }, { "cell_type": "markdown", "id": "5d7f38f2", "metadata": {}, "source": [ "Write the text again," ] }, { "cell_type": "code", "execution_count": 42, "id": "54aa0bc0", "metadata": {}, "outputs": [], "source": [ "xio.write(obj=loaded_text, path=\"quotes/most_like_with_copyright.txt\")" ] }, { "cell_type": "markdown", "id": "52002770", "metadata": {}, "source": [ "then you check that the new text includes the copyright at the bottom !" ] }, { "cell_type": "code", "execution_count": 43, "id": "eaf36478", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Nobody ever figures out what life is all about, and it doesn't matter. Explore the world. Nearly everything is really interesting if you go into it deeply enough.\r\n", "© 2022 Richard P.Feynman" ] } ], "source": [ "!cat ./quotes/most_like_with_copyright.txt" ] }, { "cell_type": "markdown", "id": "26330437", "metadata": {}, "source": [ "Finally, let's comment how to check all the registered hooks and to remove unnecessary hooks.\n", "The check is easy: just call `show_events` methods." ] }, { "cell_type": "code", "execution_count": 44, "id": "a1fe780b", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Event: post_read\n", "1. 244d96431cd5ea83: \n", "Event: post_readall\n", " No hooks are registered\n", "Event: post_write\n", " No hooks are registered\n", "Event: post_writeall\n", " No hooks are registered\n", "Event: pre_read\n", "1. 65e9f9d57134d5d7: \n", "Event: pre_readall\n", " No hooks are registered\n", "Event: pre_write\n", "1. 1269bcdeaab3db08: \n", "2. 32e966b6991c51b9: \n", "Event: pre_writeall\n", " No hooks are registered\n" ] } ], "source": [ "xio.show_events()" ] }, { "cell_type": "markdown", "id": "8824a0aa", "metadata": {}, "source": [ "Now, consider removing the hook we add at the post-reading: `remove_linebreaks`. The hook name/id is given by" ] }, { "cell_type": "code", "execution_count": 45, "id": "617c505a", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'244d96431cd5ea83'" ] }, "execution_count": 45, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# This is just temporal function to get the 1st hook id\n", "# Of course, you can take it from the above hook list and copy it\n", "def get_hook_names(event) -> list[str]:\n", " return [ hook.hook_name for hook in event.hooks ]\n", "\n", "hook_id = next(iter(get_hook_names(xio.post_read)))\n", "hook_id" ] }, { "cell_type": "code", "execution_count": 46, "id": "5f58c89b", "metadata": {}, "outputs": [], "source": [ "xio.remove_hooks(hook_id)" ] }, { "cell_type": "markdown", "id": "71742996", "metadata": {}, "source": [ "Now, it's gone as you see in the below." ] }, { "cell_type": "code", "execution_count": 47, "id": "98971cf9", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Event: post_read\n", " No hooks are registered\n", "Event: post_readall\n", " No hooks are registered\n", "Event: post_write\n", " No hooks are registered\n", "Event: post_writeall\n", " No hooks are registered\n", "Event: pre_read\n", "1. 65e9f9d57134d5d7: \n", "Event: pre_readall\n", " No hooks are registered\n", "Event: pre_write\n", "1. 1269bcdeaab3db08: \n", "2. 32e966b6991c51b9: \n", "Event: pre_writeall\n", " No hooks are registered\n" ] } ], "source": [ "xio.show_events()" ] }, { "cell_type": "markdown", "id": "feb13d8b", "metadata": {}, "source": [ "If you remove all, you can call the following method `clear_all_hooks` but it erases all the registered hooks.\n", "Note: Of course, if you restart the session, you will see the initally registered hooks when the X-I/O is loaded." ] }, { "cell_type": "code", "execution_count": 48, "id": "047d0ef8", "metadata": {}, "outputs": [], "source": [ "xio.clear_all_hooks()" ] }, { "cell_type": "code", "execution_count": 49, "id": "1b5c478c", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Event: post_read\n", " No hooks are registered\n", "Event: post_readall\n", " No hooks are registered\n", "Event: post_write\n", " No hooks are registered\n", "Event: post_writeall\n", " No hooks are registered\n", "Event: pre_read\n", " No hooks are registered\n", "Event: pre_readall\n", " No hooks are registered\n", "Event: pre_write\n", " No hooks are registered\n", "Event: pre_writeall\n", " No hooks are registered\n" ] } ], "source": [ "xio.show_events()" ] }, { "cell_type": "markdown", "id": "76c6704f", "metadata": {}, "source": [ "Now, you did a great job and know our package well !" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.1" }, "toc": { "base_numbering": 1, "nav_menu": {}, "number_sections": true, "sideBar": true, "skip_h1_title": false, "title_cell": "Table of Contents", "title_sidebar": "Contents", "toc_cell": false, "toc_position": { "height": "calc(100% - 180px)", "left": "10px", "top": "150px", "width": "340.083px" }, "toc_section_display": true, "toc_window_display": true } }, "nbformat": 4, "nbformat_minor": 5 }