python logging - different level for specific function

Question

i'm trying to reduce the amount of logging that the napalm library sends to syslog, but also allow for info logs to be sent from other parts of the code. I set up logging.basicConfig to be INFO but then i'd like the napalm function to be WARNING and above.

So i have code like this:

from napalm import get_network_driver
import logging
import getpass

logging.basicConfig(
    filename="/var/log/myscripts/script.log", level=logging.INFO, format="%(asctime)s %(message)s")

def napalm(device):
    logging.getLogger().setLevel(logging.WARNING)
    username = getpass.getuser()
    driver = get_network_driver("junos")
    router = driver(str(device), username, "", password="", timeout=120)
    router.open()
    return router

router = napalm('myrouter')
config = "hostname foobar"
router.load_merge_candidate(config=config)
show = router.compare_config()
logging.info(show)

The issue is the logging.info output never makes it to the log file. If i do logging.warning(show) it does, but i'd like this to be info. The reason i want the function to be WARNING is that it generates so much other logging at the info level that is just noise. So trying to cut down on that.

Setting the log level is "sticky". Why not just set back to `INFO` at the end of the function? — 0x5453, Jan 29 '21 at 16:45
Does this answer your question? [Turn off logging in schedule library](https://stackoverflow.com/questions/38102291/turn-off-logging-in-schedule-library) — MisterMiyagi, Jan 29 '21 at 16:50
@0x5453 oh, so i could just set `logging.getLogger().setLevel(logging.INFO)` at the end of the function? — PhillipCres, Jan 29 '21 at 17:04

score 3 · Answer 1 · answered Jan 29 '21 at 17:14

3

A nice trick from the book Effective Python. See if it helps your situation.

def napalm(count):
    for x in range(count):
        logger.info('useless log line')

@contextmanager
def debug_logging(level):
    logger = logging.getLogger()
    old_level = logger.getEffectiveLevel()
    logger.setLevel(level)
    try:
        yield
    finally:
        logger.setLevel(old_level)

napalm(5)
with debug_logging(logging.WARNING):
    napalm(5)
napalm(5)

answered Jan 29 '21 at 17:14

lllrnr101

2,288
2
4
15

Since the logger level is shared by all threads, wouldn't this be a bad idea in a multithreaded environment? You may end up logging things you don't mean to, or worse, lose logs you were supposed to capture. EDIT: I see that the behavior is different if you use the root logger (thread unsafe) as opposed to getting a custom logger (seems to be thread safe). – Shmuel Kamensky Jan 07 '22 at 16:51

Czaporka · Accepted Answer · 2021-01-30T10:13:30.790

By calling logging.getLogger() without a parameter you are currently retrieving the root logger, and overriding the level for it also affects all other loggers. You should instead retrieve the library's logger and override level only for that specific one.

The napalm library executes the following in its __init__.py:

logger = logging.getLogger("napalm")

i.e. the library logger's name is "napalm". You should thus be able to override the level of that specific logger by putting the following line in your script:

logging.getLogger("napalm").setLevel(logging.WARNING)

Generic example:

import logging

logging.basicConfig(level=logging.DEBUG, format="%(levelname)s: %(message)s")
A = logging.getLogger("A")
B = logging.getLogger("B")

A.debug("#1 from A gets printed")
B.debug("#1 from B gets printed")

logging.getLogger("A").setLevel(logging.CRITICAL)

A.debug("#2 from A doesn't get printed")  # because we've increased the level
B.debug("#2 from B gets printed")

Output:

DEBUG: #1 from A gets printed
DEBUG: #1 from B gets printed
DEBUG: #2 from B gets printed

Edit:

Since this didn't work well for you, it's probably because there's a bunch of various other loggers in this library:

$ grep -R 'getLogger' .
napalm/junos/junos.py:log = logging.getLogger(__file__)
napalm/base/helpers.py:logger = logging.getLogger(__name__)
napalm/base/clitools/cl_napalm.py:logger = logging.getLogger("napalm")
napalm/base/clitools/cl_napalm_validate.py:logger = logging.getLogger("cl_napalm_validate.py")
napalm/base/clitools/cl_napalm_test.py:logger = logging.getLogger("cl_napalm_test.py")
napalm/base/clitools/cl_napalm_configure.py:logger = logging.getLogger("cl-napalm-config.py")
napalm/__init__.py:logger = logging.getLogger("napalm")
napalm/pyIOSXR/iosxr.py:logger = logging.getLogger(__name__)
napalm/iosxr/iosxr.py:logger = logging.getLogger(__name__)

I would then resort to something like this (related: How to list all existing loggers using python.logging module):

# increase level for all loggers
for name, logger in logging.root.manager.loggerDict.items():
    logger.setLevel(logging.WARNING)

or perhaps it will suffice if you just print the various loggers, identify the most noisy ones, and silence them one by one.

hmm trying this without luck.. would i need to change what i'm importing from napalm? — PhillipCres, Jan 29 '21 at 18:46
@PhillipCres Can you please add the imports to the question? I'll then try to figure out. — Czaporka, Jan 29 '21 at 18:48
@PhillipCres I've edited the answer, try out the `for` loop. — Czaporka, Jan 29 '21 at 19:24
thanks!! Turned out it was a totally different logger causing all the noise. I was able to set that specific one to WARN using the `logging.root.manager.loggerDict.items()` approach. — PhillipCres, Jan 29 '21 at 21:38

python logging - different level for specific function

2 Answers2

Generic example:

Edit: