Ufdjrw

What's the difference between `auto x = vector<int>()` and `vector<int> x`?

Does surprise arrest existing movement?

If 'B is more likely given A', then 'A is more likely given B'

Why aren't air breathing engines used as small first stages

Should I call the interviewer directly, if HR aren't responding?

How to recreate this effect in Photoshop?

What do you call a plan that's an alternative plan in case your initial plan fails?

What causes the vertical darker bands in my photo?

How to motivate offshore teams and trust them to deliver?

What are the pros and cons of Aerospike nosecones?

Did Xerox really develop the first LAN?

3 doors, three guards, one stone

Can a non-EU citizen traveling with me come with me through the EU passport line?

When -s is used with third person singular. What's its use in this context?

What is the correct way to use the pinch test for dehydration?

How to find all the available tools in macOS terminal?

How can players work together to take actions that are otherwise impossible?

Why is there no army of Iron-Mans in the MCU?

Should I discuss the type of campaign with my players?

How much radiation do nuclear physics experiments expose researchers to nowadays?

Is there a service that would inform me whenever a new direct route is scheduled from a given airport?

Is the address of a local variable a constexpr?

How can I fade player when goes inside or outside of the area?

Why was the term "discrete" used in discrete logarithm?

Change the logger one will get when asking for root

.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty height:90px;width:728px;box-sizing:border-box;

0

I'm using pythons logging utility for logging in multiple threads (concurrent). Unlike many examples on this (cookbook, another so question) I want each thread to log to its own file (keep the thread-logs separate). This works well by using a separate FileHandler for each logger (and by explicitly getting a named logger for each thread) as shown in the below example.

Now the problem arises that I'd like to call more generic packages from within each thread and also from other places in the code that produce their owns loggings. How do I ensure their loggings are captured in the file belonging to the thread that called it?

import logging
import concurrent.futures

def generic_function(xyz):
 logger = logging.getLogger(__name__)
 logger.info("Very generic stuff")

def do_work(i):
 logger = logging.getLogger(f'loggeri')
 logger.setLevel(logging.INFO)
 fh = logging.FileHandler(f'logging_i.log')
 logger.addHandler(fh)

 logger.info(f"BeeBaaBoo i")
 generic_function(123)

 return 

outps = []
with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor:
 for i in range(10):
 outp = executor.submit(do_work, i)
 outps.append(outp)

for outp in outps:
 if outp.exception() is not None:
 raise outp.exception()

I know I can pass around the logger instance or its name, but that would require me to make the generic package more specific. I can change the generic package to adapt better logging principles but don't want to require it to expect my threading business.

Maybe it's possible to change the logger one will get when asking for a new one?

python multithreading logging concurrent.futures

share|improve this question

edited Mar 8 at 16:32

Bastiaan

asked Mar 8 at 16:25

BastiaanBastiaan

2,03311523

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  I could see putting each work item's logs in a separate file, but why would you want to segregate logs by thread? If you really want to do that, it should be the responsibility of the log Handler rather than requiring a separate Logger object for each thread.
  
  – Daniel Pryden
  Mar 8 at 16:38
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  Not sure if I understand your question. One work thread calculates everything about a human protein, its binding site, its moleculary interaction field, its associated ligand bindings etc. The next thread does the same thing, for another protein. During these calculations there are all sorts of things that work and don't work, I'd like to keep a track of that per protein. If I send this all to the same file, output from different protein's will be all interleaved. Still think I shouldn't want this or do it differently? I'd like to hear,...
  
  – Bastiaan
  Mar 8 at 16:46
  
  

  
  
  
  


• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  
  My point is that you don't need a separate Logger to get separate output. A Logger is a source of log messages; a Handler is what produces output. If you want to define a context for a series of log messages that are related (so that a Handler can do something different with them), you probably want to use a Filter that reads from a thread-local variable.
  
  – Daniel Pryden
  Mar 8 at 18:11
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  Ah, that could be it! And the filter can be different for the same library depending on where it got called from?
  
  – Bastiaan
  Mar 8 at 19:20
  

  
  
  
  

add a comment | 

0

I'm using pythons logging utility for logging in multiple threads (concurrent). Unlike many examples on this (cookbook, another so question) I want each thread to log to its own file (keep the thread-logs separate). This works well by using a separate FileHandler for each logger (and by explicitly getting a named logger for each thread) as shown in the below example.

Now the problem arises that I'd like to call more generic packages from within each thread and also from other places in the code that produce their owns loggings. How do I ensure their loggings are captured in the file belonging to the thread that called it?

import logging
import concurrent.futures

def generic_function(xyz):
 logger = logging.getLogger(__name__)
 logger.info("Very generic stuff")

def do_work(i):
 logger = logging.getLogger(f'loggeri')
 logger.setLevel(logging.INFO)
 fh = logging.FileHandler(f'logging_i.log')
 logger.addHandler(fh)

 logger.info(f"BeeBaaBoo i")
 generic_function(123)

 return 

outps = []
with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor:
 for i in range(10):
 outp = executor.submit(do_work, i)
 outps.append(outp)

for outp in outps:
 if outp.exception() is not None:
 raise outp.exception()

I know I can pass around the logger instance or its name, but that would require me to make the generic package more specific. I can change the generic package to adapt better logging principles but don't want to require it to expect my threading business.

Maybe it's possible to change the logger one will get when asking for a new one?

python multithreading logging concurrent.futures

share|improve this question

edited Mar 8 at 16:32

Bastiaan

asked Mar 8 at 16:25

BastiaanBastiaan

2,03311523

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  I could see putting each work item's logs in a separate file, but why would you want to segregate logs by thread? If you really want to do that, it should be the responsibility of the log Handler rather than requiring a separate Logger object for each thread.
  
  – Daniel Pryden
  Mar 8 at 16:38
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  Not sure if I understand your question. One work thread calculates everything about a human protein, its binding site, its moleculary interaction field, its associated ligand bindings etc. The next thread does the same thing, for another protein. During these calculations there are all sorts of things that work and don't work, I'd like to keep a track of that per protein. If I send this all to the same file, output from different protein's will be all interleaved. Still think I shouldn't want this or do it differently? I'd like to hear,...
  
  – Bastiaan
  Mar 8 at 16:46
  
  

  
  
  
  


• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  
  My point is that you don't need a separate Logger to get separate output. A Logger is a source of log messages; a Handler is what produces output. If you want to define a context for a series of log messages that are related (so that a Handler can do something different with them), you probably want to use a Filter that reads from a thread-local variable.
  
  – Daniel Pryden
  Mar 8 at 18:11
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  Ah, that could be it! And the filter can be different for the same library depending on where it got called from?
  
  – Bastiaan
  Mar 8 at 19:20
  

  
  
  
  

add a comment | 

I'm using pythons logging utility for logging in multiple threads (concurrent). Unlike many examples on this (cookbook, another so question) I want each thread to log to its own file (keep the thread-logs separate). This works well by using a separate FileHandler for each logger (and by explicitly getting a named logger for each thread) as shown in the below example.

Now the problem arises that I'd like to call more generic packages from within each thread and also from other places in the code that produce their owns loggings. How do I ensure their loggings are captured in the file belonging to the thread that called it?

import logging
import concurrent.futures

def generic_function(xyz):
 logger = logging.getLogger(__name__)
 logger.info("Very generic stuff")

def do_work(i):
 logger = logging.getLogger(f'loggeri')
 logger.setLevel(logging.INFO)
 fh = logging.FileHandler(f'logging_i.log')
 logger.addHandler(fh)

 logger.info(f"BeeBaaBoo i")
 generic_function(123)

 return 

outps = []
with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor:
 for i in range(10):
 outp = executor.submit(do_work, i)
 outps.append(outp)

for outp in outps:
 if outp.exception() is not None:
 raise outp.exception()

I know I can pass around the logger instance or its name, but that would require me to make the generic package more specific. I can change the generic package to adapt better logging principles but don't want to require it to expect my threading business.

Maybe it's possible to change the logger one will get when asking for a new one?

python multithreading logging concurrent.futures

share|improve this question

edited Mar 8 at 16:32

Bastiaan

asked Mar 8 at 16:25

BastiaanBastiaan

2,03311523

import logging
import concurrent.futures

def generic_function(xyz):
 logger = logging.getLogger(__name__)
 logger.info("Very generic stuff")

def do_work(i):
 logger = logging.getLogger(f'loggeri')
 logger.setLevel(logging.INFO)
 fh = logging.FileHandler(f'logging_i.log')
 logger.addHandler(fh)

 logger.info(f"BeeBaaBoo i")
 generic_function(123)

 return 

outps = []
with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor:
 for i in range(10):
 outp = executor.submit(do_work, i)
 outps.append(outp)

for outp in outps:
 if outp.exception() is not None:
 raise outp.exception()

share|improve this question

edited Mar 8 at 16:32

Bastiaan

asked Mar 8 at 16:25

BastiaanBastiaan

2,03311523

share|improve this question

edited Mar 8 at 16:32

Bastiaan

asked Mar 8 at 16:25

BastiaanBastiaan

2,03311523

asked Mar 8 at 16:25

BastiaanBastiaan

2,03311523

asked Mar 8 at 16:25

BastiaanBastiaan

2,03311523