CUPS accounting [Solved]

Altoid · 2020-02-17 19:55:25

Hello:

I'm looking for a way to quickly be able to know how many pages my Samsung M2020W has printed at any point in time.
Although not really at any point, just when I see that the print density is falling.

The original cartdrige lasted for few pages (I know that is the usual thing) and I have had it refilled and would like to keep a tab on how well it works.

There's the log at http://localhost:631/admin/log/page_log which is read from /var/log/cups/page_log but I don't see how to weed out the data without some time consuming editing.

Thanks in advance,

A.

Last edited by Altoid (2020-03-29 20:42:12)

ToxicExMachina · 2020-02-18 07:17:24

Try too find it in completed jobs of the printer (CUPS web interface). You will see table with information you need.

Altoid · 2020-02-18 10:43:14

Hello:

ToxicExMachina wrote:

.... in completed jobs of the printer ...

Yes, I know.

Like I mention in my post:

There's the log at http://localhost:631/admin/log/page_log which is read from /var/log/cups/page_log ...

But there are more than 400 completed jobs and I only need the amount of pages printed which I cannot seem to be able to get without some time consuming editing of the log file.

Thanks for your input.

A.

ToxicExMachina · 2020-02-21 07:45:36

Altoid wrote:

Hello:
ToxicExMachina wrote:
.... in completed jobs of the printer ...
Yes, I know.
Like I mention in my post:
There's the log at http://localhost:631/admin/log/page_log which is read from /var/log/cups/page_log ...
But there are more than 400 completed jobs and I only need the amount of pages printed which I cannot seem to be able to get without some time consuming editing of the log file.
Thanks for your input.
A.

You can use grep and/or awk for getting only necessary information from that log file.

Altoid · 2020-02-23 00:01:48

Hello:

ToxicExMachina wrote:

... use grep and/or awk for getting only necessary information from that log file.

Hmmm ...

The data I'm looking for is the sum of a set of numbers whose values I ignore and are (in this case) listed in 400 consecutive lines, one line per print job.
I don't see how grep could help me.

I thought there could be some script or front end I have not been able to find that would use the log file as input and get the total pages printed.

Thanks for your input.

Cheers,

A.

ToxicExMachina · 2020-02-23 11:31:53

Altoid wrote:

Hello:
ToxicExMachina wrote:
... use grep and/or awk for getting only necessary information from that log file.
Hmmm ...
The data I'm looking for is the sum of a set of numbers whose values I ignore and are (in this case) listed in 400 consecutive lines, one line per print job.
I don't see how grep could help me.
I thought there could be some script or front end I have not been able to find that would use the log file as input and get the total pages printed.
Thanks for your input.
Cheers,
A.

May be I can help but I need a sample of log. I am not sure /var/log/cups/page_log is the correct log because in my case it has no any contents (the machine with working printer).

Altoid · 2020-02-23 12:40:01

Hello:

ToxicExMachina wrote:

Maybe I can help ...

Thank you very much. =-)

ToxicExMachina wrote:

... but I need a sample of log.
... not sure /var/log/cups/page_log is the correct log ...
... it has no any contents ...

It seems that CUPS can be quite informative.
I have three logs: access_log, error_log, and page_log, but I think logging/log level has (?) to be set in the configuration file.

If I remember correctly, I'm using the default file which you can set in http://localhost:631/admin/.
The first entry is LogLevel and it is set to warn.

Also, in http://localhost:631/admin/ -> Server Settings -> Advanced you can choose to preserve job history.
I have it set to 50 jobs max.

With respect to the page_log format, I found this ...

Page Logging
CUPS can log every page that is printed on a system to the page_log file. Page logging must be enabled by setting the PageLogFormat directive in the cupsd.conf file and is only available for drivers that provide page accounting information, typically all PostScript and CUPS raster devices. Raw queues and queues using third-party solutions such as Foomatic generally do not have useful page accounting information available.

... and this:

Access Log File Format
The access_log file lists each HTTP resource that is accessed by a web browser or client. Each line is in an extended version of the so-called "Common Log Format" used by many web servers and web reporting tools:
host group user date-time "method resource version" status bytes ipp-operation ipp-status

It seems that the default page_log format (apparently set in cupsd.conf) is this ...

PageLogFormat format-string
Specifies the format of PageLog lines. Sequences beginning with percent (%) characters are replaced with the corresponding information, while all other characters are copied literally. The following percent sequences are recognized:
"%%" inserts a single percent character.
"%{name}" inserts the value of the specified IPP attribute.
"%C" inserts the number of copies for the current page.
"%P" inserts the current page number.
"%T" inserts the current date and time in common log format.
"%j" inserts the job ID.
"%p" inserts the printer name.
"%u" inserts the username.
The default is the empty string, which disables page logging. The string "%p %u %j %T %P %C %{job-billing} %{job-originating-host-name} %{job-name} %{media} %{sides}" creates a page log with the standard items.

... but my /etc/cups/cupsd.conf dos not have a PageLogFormat entry.

In any case, the /var/log/cups/page_log format in my log file is this:

Samsung_M2020_Series groucho 109 [27/Dec/2018:08:37:02 -0300] 1 1 - localhost H:\Desktop\flower_sm.jpg A4 -

I have to see how to trim this a bit as I really have no use for so much information as (at least for now) it is one rig, one user and no external sharing or network access and I only need to know how many pages I have been able to print by the time my cheap HP printer with an expensive toner cartridge needs to be reloaded.

Thanks a lot for your input.

Cheers,

A.

ToxicExMachina · 2020-02-24 09:14:27

The simplest way to trim information from the log is awk

I suppose in the "Samsung_M2020_Series groucho 109 [27/Dec/2018:08:37:02 -0300] 1 1 - localhost H:\Desktop\flower_sm.jpg A4 -" entry the 6th word (in this case it's "1") is the number of printed pages. Therefore the following command will give a text table with date/time and amount of printed pages found by the word "A4":

cat /var/log/cups/page_log | grep A4 | awk '{print $4$5 " " $6}' > trimmed_log.txt

You can process it in spreadsheet.

If you want to get total amount of pages printed by the printer you can use the following command:

cat /var/log/cups/page_log | grep A4 | awk '{pages += $6} END {print pages}'

In this simple example grep is a tool used to find log entries according to defined word and awk is a tool used to parse and write every entry found by grep in shorter format. The default field separator is space. You can modify expressions from the example for best result.

Last edited by ToxicExMachina (2020-02-24 09:14:41)

Head_on_a_Stick · 2020-02-24 18:42:03

ToxicExMachina wrote:

Therefore the following command will give a text table with date/time and amount of printed pages found by the word "A4":
cat /var/log/cups/page_log | grep A4 | awk '{print $4$5 " " $6}' > trimmed_log.txt 
You can process it in spreadsheet.
If you want to get total amount of pages printed by the printer you can use the following command:
cat /var/log/cups/page_log | grep A4 | awk '{pages += $6} END {print pages}'

Useless use of cat. And grep too.

Try these instead:

awk '/A4/{print $4$5 " " $6}' /var/log/cups/page_log > trimmed_log.txt
awk '/A4/{pages += $6} END {print pages}' /var/log/cups/page_log

Altoid · 2020-02-25 12:32:40

Hello:

Head_on_a_Stick wrote:

Try these instead ...

These two worked a charm.

This string awk '/A4/{print $4$5 " " $6}' /var/log/cups/page_log > trimmed_log.txt got me a list of what would seem to be all the print jobs up to today (172 between 30/11/18 and 13/02/2020). I'd just have to trim the ones up to the cartdrige change/refill.

groucho@devuan:~$ cat trimmed_log.txt
[30/Nov/2018:10:44:10-0300] 1
[08/Dec/2018:19:12:32-0300] 1
--- snip ---
13/Feb/2020:16:57:53-0300] 14
groucho@devuan:~$

And this other one awk '/A4/{pages += $6} END {print pages}' /var/log/cups/page_log gave me what would seem to be the sum of the pages printed (601).

groucho@devuan:~$ awk '/A4/{pages += $6} END {print pages}' /var/log/cups/page_log
601
groucho@devuan:~$

No spreadsheet, which is a relief even though I would only do this once every so often.

Thank you both for taking the time to help me out with this, I will try harder to spruce up my CLI basics as I am (again, after so long) finding it to be not only interesting but very usefull.
So many years of GUI usage ... =-/

Once again, thanks a lot.

Best,

A.

Edit:
There seem to be some discrepancies between the actual log and the output from the awk commands.
I'll check and report back.

Last edited by Altoid (2020-02-25 12:58:23)

Altoid · 2020-02-26 12:25:30

Hello:

Altoid wrote:

Hello:
There seem to be some discrepancies ...
I'll check and report back.

I've done some manual accounting and found this:

While the output of the awk command to find the total pages printed gives a result of 601 ...

groucho@devuan:~$ awk '/A4/{pages += $6} END {print pages}' /var/log/cups/page_log
601
groucho@devuan:~$

... my manual accounting shows that a total of 1033 pages were actually printed.

The 1033 (via-a-vis 601) figure is more in line with the expected number of pages that could actually be printed with the original cartdrige that came with this HP printer.
(cheap bastards ... )

This is the output of the relevant parts of /var/log/cups/page_log:

groucho@devuan:~$ cat /var/log/cups/page_log
--- snip ---
Samsung_M2020_Series groucho 84 [02/Nov/2018:10:50:45 -0300] 1 1 - localhost
Samsung_M2020_Series groucho 85 [08/Nov/2018:18:10:50 -0300] 1 1 - localhost
Samsung_M2020_Series groucho 86 [27/Nov/2018:16:41:51 -0300] 1 1 - localhost
Samsung_M2020_Series groucho 87 [30/Nov/2018:10:44:10 -0300] 1 1 - localhost
Samsung_M2020_Series groucho 88 [05/Dec/2018:08:29:39 -0300] 1 1 - localhost
Samsung_M2020_Series groucho 88 [05/Dec/2018:08:29:42 -0300] 2 1 - localhost
Samsung_M2020_Series groucho 89 [08/Dec/2018:19:12:32 -0300] 1 1 - localhost 
--- snip ---

For some reason the print job numbering starts at 84, which at 1 page per job, would take the number of printed pages to 1117.
I may have deleted/edited the log file after finding strange characters in some lines due to some ISO issue (?), cannot recall.

I've struck out the data after localhost as for my use I have no need for the printed page metadata nor the page size as I think this printer only takes A4, the only size I am using.
I have no need for the printer data or the user data either as there's only one printer and only one user allowed to print or use the rig (me) but I have left that part of the output string to show how it is formed.

Samsung_M2020_Series
groucho
84 <--- job number
[02/Nov/2018:10:50:45 -0300] <--- job date
1 <--- pages printed
1 <--- pages in job
localhost

The thing is that only the data in bold is of any use to me.

What would have to be added/changed to the awk stanza to get this to work?
I have the idea that whatever the working awk stanza is, it will be good for this particular setup as a change in the printer name or user could throw off the results.

Is this so?

Thanks in advance,

A.

Last edited by Altoid (2020-03-19 11:58:55)

Altoid · 2020-03-19 12:43:50

Hello:

Altoid wrote:

... whatever the working awk stanza is, it will be good for this particular setup as a change in the printer name or user could throw off the results.
Is this so?

It seems that it is more compliacted than that.

For my purposes (printed page accounting) the relevant data in each line of the /var/log/cups/page_log file boils down to the number that tells me the amount of pages printed in each print job.

As there's only one printer Samsung_M2020_Series and only one user groucho the column where the print job number, page number and pager printed data are will always be in the same position.

Data that comes after that informing page size and content is irrelevant for what I need to do.

Samsung_M2020_Series groucho 1 [02/Nov/2018:10:50:45 -0300] 1 1 - localhost
Samsung_M2020_Series groucho 2 [08/Nov/2018:18:10:50 -0300] 1 1 - localhost
Samsung_M2020_Series groucho 3 [27/Nov/2018:16:41:51 -0300] 1 1 - localhost
Samsung_M2020_Series groucho 3 [27/Nov/2018:16:42:00 -0300] 2 1 - localhost
Samsung_M2020_Series groucho 4 [05/Dec/2018:08:29:39 -0300] 1 1 - localhost
                             | <- print job  page number -> | | <- pages printed

So if I run awk '/groucho/{pages += $7} END {print pages}' against that data, it correctly tells me how many pages I have printed.
ie: 5 pages.

Now, when printing starts, print job will be a one digit number for the first nine lines in the log and from then on, will be two digit numbers for the next 99 and so on.

Page number and pages printed will also be one digit numbers as long as the print job being logged has only nine different pages to print and the amount of pages printed is under 10.

But when these change, the awk stanza is useless.

Even if I trim the data being logged (I'm on that) to limit it to just the bare essentials (date, page number and pages printed), the problem susbsists as these last two will change.

How to get around that?

Thanks in advance,

A.

Last edited by Altoid (2020-03-19 12:46:30)

chris2be8 · 2020-03-19 17:30:57

awk counts words separated by spaces. So as long as the number of pages printed is just a string of digits awk will total it correctly. I've run similar code to total multi digit numbers without problems.

You could make a copy of some of the log, edit the numbers, run awk against it and see what you get. That should find any problems in time to fix them

Chris

Last edited by chris2be8 (2020-03-19 17:31:14)

Altoid · 2020-03-19 22:19:03

Hello:

chris2be8 wrote:

awk counts words separated by spaces. So as long as the number of pages printed is just a string of digits ...

Yes, but ...

Samsung_M2020_Series groucho 3 [27/Nov/2018:16:41:51 -0300] 1 1 - localhost
Samsung_M2020_Series groucho 3 [27/Nov/2018:16:42:00 -0300] 2 1 - localhost
Samsung_M2020_Series groucho 4 [05/Dec/2018:08:29:39 -0300] 1 1 - localhost
                             | <- print job  page number -> | | <- pages printed

In the example above, we see two print jobs: print job 3 which outputs 1 copy of page 1 and 1 copy of page 2 and print job 4, which outputs 1 copy of page 1.
The awk stanza in my previous post will correctly tally the amount of pages printed, ie: 3 pages.

groucho@devuan:~/Desktop/awkcups$ awk '/groucho/{pages += $7} END {print pages}' /home/groucho/Desktop/awkcups/plog1
3
groucho@devuan:~/Desktop/awkcups$

Now, in a different scenario, where we have two digit print jobs with the rest being the same ...

Samsung_M2020_Series groucho 13 [27/Nov/2018:16:41:51 -0300] 1 1 - localhost
Samsung_M2020_Series groucho 13 [27/Nov/2018:16:42:00 -0300] 2 1 - localhost
Samsung_M2020_Series groucho 14 [05/Dec/2018:08:29:39 -0300] 1 1 - localhost

... the awk stanza previously applied gives an incorrect output, ie: 5 pages

groucho@devuan:~/Desktop/awkcups$ awk '/groucho/{pages += $7} END {print pages}' /home/groucho/Desktop/awkcups/pg1log
5
groucho@devuan:~/Desktop/awkcups$

The same problem would arise if a print job is for 20 copies of a single page.

At the risk of stating the obvious, the same awk stanza parses the /var/log/cups/page_log file which (depending on how the printer is used) can have between one and four digits both in the print job number and the number of pages printed. I limit this to four digits because the size of cartdrige 2020W cartdrige won't ever print 9999 pages.

I can't see how to solve this with awk and have searched all over the web for a script to do basic CUPS accounting to no avail.

Thanks for your input.

A.

chris2be8 · 2020-03-20 18:22:21

I've just tried copying the two sample logs you gave above and running awk against them and get 3 both times:

chris@rigel:~/ecm-3031$ cat cupslog1 
Samsung_M2020_Series groucho 3 [27/Nov/2018:16:41:51 -0300] 1 1 - localhost
Samsung_M2020_Series groucho 3 [27/Nov/2018:16:42:00 -0300] 2 1 - localhost
Samsung_M2020_Series groucho 4 [05/Dec/2018:08:29:39 -0300] 1 1 - localhost
chris@rigel:~/ecm-3031$ awk '/groucho/{pages += $7} END {print pages}' cupslog1 
3
chris@rigel:~/ecm-3031$ cat cupslog2 
Samsung_M2020_Series groucho 13 [27/Nov/2018:16:41:51 -0300] 1 1 - localhost
Samsung_M2020_Series groucho 13 [27/Nov/2018:16:42:00 -0300] 2 1 - localhost
Samsung_M2020_Series groucho 14 [05/Dec/2018:08:29:39 -0300] 1 1 - localhost
chris@rigel:~/ecm-3031$ awk '/groucho/{pages += $7} END {print pages}' cupslog2 
3

So I can't see what's going wrong for you.

Chris

Altoid · 2020-03-20 20:17:58

Hello:

chris2be8 wrote:

... running awk against them and get 3 both times
... can't see what's going wrong for you.

Hmm ...
I copied and pasted the output from the terminal so I cannot see what happened either.

But you are quite correct:
Running that awk stanza on this specific /var/log/cups/page_log configuration will tally the printed pages correctly.
I summed them up by hand and it checked out. =-)

Turns out that the OEM cartdrige that came with the printer ended up being good for just 362 pages:

groucho@devuan:~/Desktop/awkcups$ awk '/groucho/{pages += $7} END {print pages}' /var/log/cups/page_log
362
groucho@devuan:~/Desktop/awkcups$

I now have to see how to go about trimming the page_log output, no need for all that data.

Thank you (all) very much for your help with this.

Cheers,

A.

The officially official Devuan Forum!

#1 2020-02-17 19:55:25

CUPS accounting [Solved]

#2 2020-02-18 07:17:24

Re: CUPS accounting [Solved]

#3 2020-02-18 10:43:14

Re: CUPS accounting [Solved]

#4 2020-02-21 07:45:36

Re: CUPS accounting [Solved]

#5 2020-02-23 00:01:48

Re: CUPS accounting [Solved]

#6 2020-02-23 11:31:53

Re: CUPS accounting [Solved]

#7 2020-02-23 12:40:01

Re: CUPS accounting [Solved]

#8 2020-02-24 09:14:27

Re: CUPS accounting [Solved]

#9 2020-02-24 18:42:03

Re: CUPS accounting [Solved]

#10 2020-02-25 12:32:40

Re: CUPS accounting [Solved]

#11 2020-02-26 12:25:30

Re: CUPS accounting [Solved]

#12 2020-03-19 12:43:50

Re: CUPS accounting [Solved]

#13 2020-03-19 17:30:57

Re: CUPS accounting [Solved]

#14 2020-03-19 22:19:03

Re: CUPS accounting [Solved]

#15 2020-03-20 18:22:21

Re: CUPS accounting [Solved]

#16 2020-03-20 20:17:58

Re: CUPS accounting [Solved]

Board footer