Subject: devel-digest V2 #186
From: [email-masked] (devel-digest)
To: [email-masked]

devel-digest          Wednesday, June 2 1999          Volume 02 : Number 186

In this issue:

    memory bandwidth problems
    Re: memory bandwidth problems
    Re: memory bandwidth problems
    Re: memory bandwidth problems
    Re: memory bandwidth problems
    Re: memory bandwidth problems
    Re: memory bandwidth problems
    Re: memory bandwidth problems
    Re: memory bandwidth problems

See the end of this digest for information about devel-digest.

----------------------------------------------------------------------

Date: Tue, 1 Jun 1999 12:05:30 -0700
From: Tim Rowledge <[email-masked]>
Subject: memory bandwidth problems

The thread seems to have faded duringthe transition to the NGs, but  I would
like to understand what is happening here. The following trivial program
measures the effective bandwidth of memory access in a way that is moderately
related to what I need to do in my App.

#include <stdio.h>
int main() {
char * p;
int i, j, t0, imSize=704*480*2;
p = (char*)malloc(imSize);
t0 = clock();
for( j=0; j<1000; j++) {
	for (i = 0; i < imSize; i++) {
		p[i] +=1;
	}
	for (i = 0; i < imSize; i++) {
		p[i] -=1;
	}
}
printf("%d clocks for %d bytes\n", clock()-t0, 2*imSize*1000);
free(p);
}

I get '136190000 clocks for  2703360000 bytes   ' which is about 20Mb/s, or 
about 20% of what I would hope for even on a PCI bus! As a contrast my five
year old Acorn RPc witha 202MHz SA-100 and a 16MHz memory bus runs the same
program in 88.4sec for about 30Mb/s bandwidth. _Something_ is wrong here.

tim

- ---
This message processed by the Netwinder.Org news2mail gateway.

- --
This list is run automagically by Majordomo 1.94.4
Please send email questions to [email-masked]
To UNSUBSCRIBE: email [email-masked] with
"unsubscribe devel" as the body of your message.

------------------------------

Date: Tue, 01 Jun 1999 21:27:52 +0000
From: Mike Montour <[email-masked]>
Subject: Re: memory bandwidth problems

Tim Rowledge wrote:

>int i, j, t0, imSize=704*480*2

> printf("%d clocks for %d bytes\n", clock()-t0, 2*imSize*1000);

> I get '136190000 clocks for  2703360000 bytes   ' which is about 20Mb/s,
> about 20% of what I would hope for even on a PCI bus! 

I'm not sure how you got "2703360000 bytes" from the program
listing you supplied. By my math, 

  imSize=704*480*2 = 675840

  2*imSize*1000 = 2*(675840)*1000 = 1351680000 (off by a factor of 2)

However, that is not an issue for relative comparisons. 

I modified your program for 100 iterations rather than 1000, and ran
it on my Netwinder and an x86 (Pentium-233 I think, though it might 
be a 200 (466.94 bogomips on Linux-2.0.32)). Results are:

Netwinder, default compile:  13270000 clocks for 135168000 bytes
x86, default compile:         9950000 clocks for 135168000 bytes 
Netwinder, -O2 optimization:  4530000 clocks for 135168000 bytes 
x86, -O2 optimization:        3770000 clocks for 135168000 bytes

So, although the Netwinder is slower, it's certainly in the same
ballpark as a roughly-equivalent Pentium. In both cases, there's a
large (almost 3x) speed penalty by not using compiler optimization. 

I know very little about StrongARM assembly code, so I can't 
say whether or not you could push the Netwinder even faster by
hand-tuning some parts of the loop. 

- -- 
		      -Mike  [[email-masked]]

- ---
This message processed by the Netwinder.Org news2mail gateway.

- --
This list is run automagically by Majordomo 1.94.4
Please send email questions to [email-masked]
To UNSUBSCRIBE: email [email-masked] with
"unsubscribe devel" as the body of your message.

------------------------------

Date: Tue, 01 Jun 1999 17:47:06 -0400
From: "Andrew E. Mileski" <[email-masked]>
Subject: Re: memory bandwidth problems

Tim Rowledge wrote:
> I get '136190000 clocks for  2703360000 bytes   ' which is about 20Mb/s, or
> about 20% of what I would hope for even on a PCI bus! As a contrast my five
> year old Acorn RPc witha 202MHz SA-100 and a 16MHz memory bus runs the same
> program in 88.4sec for about 30Mb/s bandwidth. _Something_ is wrong here.

Try some optimization on the program you posted.

  with no opt:
      132130000/1351680000 about  9.75 MB/sec
  with -O2 -fsigned-char -fomit-frame-pointer:
     44660000/1351680000 about 28.86 MB/sec
  with -O2 -fsigned-char -fomit-frame-pointer -funroll-loops
     30349000/1351680000 about 42.47 MB/sec

This using egcs (not that it matters for ARM).

- --
Andrew E. Mileski - Software Engineer [OS Team Leader]
Rebel.com  http://www.rebel.com/

- ---
This message processed by the Netwinder.Org news2mail gateway.

- --
This list is run automagically by Majordomo 1.94.4
Please send email questions to [email-masked]
To UNSUBSCRIBE: email [email-masked] with
"unsubscribe devel" as the body of your message.

------------------------------

Date: Tue, 1 Jun 1999 14:49:45 -0700
From: Tim Rowledge <[email-masked]>
Subject: Re: memory bandwidth problems

Mike Montour <[email-masked]> wrote:

>I'm not sure how you got "2703360000 bytes" from the program
>listing you supplied. By my math, 
>
>  imSize=704*480*2 = 675840
>
>  2*imSize*1000 = 2*(675840)*1000 = 1351680000 (off by a factor of 2)
I doubled it since each loop has to do a read/inc|dec/write, so I'd consider it
to be two bytes per loop-body. As you say , it makes no difference for
comparison, but I eventually need to work out if I can hope to do my video
capture successfully!

And...
"Andrew E. Mileski" <[email-masked]> wrote:
>Tim Rowledge wrote:
>> I get '136190000 clocks for  2703360000 bytes   ' which is about 20Mb/s, or
>> about 20% of what I would hope for even on a PCI bus! As a contrast my five
>> year old Acorn RPc witha 202MHz SA-100 and a 16MHz memory bus runs the same
>> program in 88.4sec for about 30Mb/s bandwidth. _Something_ is wrong here.
>
>Try some optimization on the program you posted.
>
>  with no opt:
>      132130000/1351680000 about  9.75 MB/sec
>  with -O2 -fsigned-char -fomit-frame-pointer:
>     44660000/1351680000 about 28.86 MB/sec
>  with -O2 -fsigned-char -fomit-frame-pointer -funroll-loops
>     30349000/1351680000 about 42.47 MB/sec
>
>This using egcs (not that it matters for ARM).
>
That's very much better - 
a) I'm a long time Acorn user and the compiler always does as many
optimizations as it can unless you wrestle it to the ground and beat it up
b) I'll see if I can do the same to the vidcap module. Maybe that'll help?

Thanks for the tips

- ---
This message processed by the Netwinder.Org news2mail gateway.

- --
This list is run automagically by Majordomo 1.94.4
Please send email questions to [email-masked]
To UNSUBSCRIBE: email [email-masked] with
"unsubscribe devel" as the body of your message.

------------------------------

Date: Tue, 01 Jun 1999 18:17:41 -0400
From: "Andrew E. Mileski" <[email-masked]>
Subject: Re: memory bandwidth problems

Tim Rowledge wrote:
> "Andrew E. Mileski" <[email-masked]> wrote:
> >Try some optimization on the program you posted.
>
> That's very much better -
> b) I'll see if I can do the same to the vidcap module. Maybe that'll help?

Perhaps.  The "-O2" code looks pretty much as the source does.
The "-O2 -funroll-loops" code is quite convoluted, but yet clever.

- --
Andrew E. Mileski - Software Engineer [OS Team Leader]
Rebel.com  http://www.rebel.com/

- ---
This message processed by the Netwinder.Org news2mail gateway.

- --
This list is run automagically by Majordomo 1.94.4
Please send email questions to [email-masked]
To UNSUBSCRIBE: email [email-masked] with
"unsubscribe devel" as the body of your message.

------------------------------

Date: Tue, 01 Jun 1999 19:57:14 -0400
From: "Andrew E. Mileski" <[email-masked]>
Subject: Re: memory bandwidth problems

Tim Rowledge wrote:
> I doubled it since each loop has to do a read/inc|dec/write, so I'd consider it
> to be two bytes per loop-body. As you say , it makes no difference for
> comparison, but I eventually need to work out if I can hope to do my video
> capture successfully!

Lets make it accurate then, before some magazine quotes these :D
The results will be double the MB of my previous post, for a total
of (with "-O2 -funroll-loops"):
  85.69 Mops/sec for bytes (char)
  62.86 Mops/sec for half words (short)
  37.26 Mops/sec for words (int)
Which is fairly interesting, assuming it means anything.
Egad, I almost forgot how much I hate benchmarks :p

#include <stdio.h>
#include <stdlib.h>
#include <time.h>

/* One of: char short int */
#define TYPE	char

/* Don't change these - they were carefully chosen */
#define REPS	256
#define OPS	(2+2)
#define SIZE	(1024*1024)
#define MOPS	(REPS*OPS)

int main() {
	TYPE *p;
	time_t start;
	unsigned elapsed, i, j;
	p = (TYPE *)malloc(SIZE * sizeof(TYPE));
	start = clock();
	for (j = 0; j < REPS; j++) {
		for (i = 0; i < SIZE; i++)
			p[i] += 1;
		for (i = 0; i < SIZE; i++)
			p[i] -= 1;
	}
	elapsed = (unsigned)(clock() - start);
	printf("%d clocks for %d ops (%f Mops/sec)\n", elapsed, MOPS*SIZE,
		((double)MOPS*CLOCKS_PER_SEC)/elapsed);
	free(p);
	return 0;
}

- --
Andrew E. Mileski - Software Engineer
Rebel.com  http://www.rebel.com/

- ---
This message processed by the Netwinder.Org news2mail gateway.

- --
This list is run automagically by Majordomo 1.94.4
Please send email questions to [email-masked]
To UNSUBSCRIBE: email [email-masked] with
"unsubscribe devel" as the body of your message.

------------------------------

Date: Tue, 01 Jun 1999 20:25:53 -0400
From: "Andrew E. Mileski" <[email-masked]>
Subject: Re: memory bandwidth problems

"Andrew E. Mileski" wrote:
> Which is fairly interesting, assuming it means anything.
> Egad, I almost forgot how much I hate benchmarks :p

Okay, here's another example of why I hate benchmarks...

In the last source I posted, the optimization is using registers as
much as possible (as one would expect).  Forcing a memory access with
volatile gives different results:
  69.28 Mops/sec for bytes (char)
  48.58 Mops/sec for half words (short)
  36.89 Mops/sec for words (int)
Notice that the word timing hasn't changed much.

#include <stdio.h>
#include <stdlib.h>
#include <time.h>

/* One of: char short int */
#define TYPE    char

/* Don't change these - they were carefully chosen */
#define REPS    256
#define OPS     (2+2)
#define SIZE    (1024*1024)
#define MOPS    (REPS*OPS)

int main() {
        volatile TYPE *p;
        time_t start;
        unsigned elapsed, i, j;
        p = (volatile TYPE *)malloc(SIZE * sizeof(TYPE));
        start = clock();
        for (j = 0; j < REPS; j++) {
                for (i = 0; i < SIZE; i++)
                        p[i] += 1;
                for (i = 0; i < SIZE; i++)
                        p[i] -= 1;
        }
        elapsed = (unsigned)(clock() - start);
        printf("%d clocks for %d ops (%f Mops/sec)\n", elapsed, MOPS*SIZE,
                ((double)MOPS*CLOCKS_PER_SEC)/elapsed);
        free((void *)p);
        return 0;
}
- --
Andrew E. Mileski - Software Engineer [OS Team Leader]
Rebel.com  http://www.rebel.com/

- ---
This message processed by the Netwinder.Org news2mail gateway.

- --
This list is run automagically by Majordomo 1.94.4
Please send email questions to [email-masked]
To UNSUBSCRIBE: email [email-masked] with
"unsubscribe devel" as the body of your message.

------------------------------

Date: Tue, 1 Jun 1999 18:15:51 -0700
From: Tim Rowledge <[email-masked]>
Subject: Re: memory bandwidth problems

"Andrew E. Mileski" <[email-masked]> wrote:
>
>Lets make it accurate then, before some magazine quotes these :D
>The results will be double the MB of my previous post, for a total
>of (with "-O2 -funroll-loops"):
>  85.69 Mops/sec for bytes (char)
>  62.86 Mops/sec for half words (short)
>  37.26 Mops/sec for words (int)
>Which is fairly interesting, assuming it means anything.
>Egad, I almost forgot how much I hate benchmarks :p
>
Me too. Everyone wants to reduce complex things to a single number and then use
it to make complex multi-variant decisions. How fast can your car do 0-60? Wow,
mine is better than yours! forgetting of course that my bike can do 0-100 in
6sec...

In this case I'm being narrowly focussed; it was a test to see if there
seemed to be anything wrong with memory. Although I can't get the same results
as you (my best time with chars is ~38secs rather than ~30) I _can_ get ~154Mb/s
with word ops, which is pretty good. Sadly, recompiling vidcap.o with
- -funroll-loops makes no improvements :-( I can still only get 10fps of
fullframe video. Poo.

One possible improvement in overall performance I might be able to make use of
is incorporating one of the compression passes into the grab -after all, I have
to read the memory, fiddle with  it and write it, so why not read from
video memory and write it into user memory at the same time? Are there any
'odd' things going on in copy_to_user() that would prevent this from being
feasible? It looks like it should simply be amemcpy() loop.

- ---
This message processed by the Netwinder.Org news2mail gateway.

- --
This list is run automagically by Majordomo 1.94.4
Please send email questions to [email-masked]
To UNSUBSCRIBE: email [email-masked] with
"unsubscribe devel" as the body of your message.

------------------------------

Date: Tue, 01 Jun 1999 22:35:05 -0400
From: "Andrew E. Mileski" <[email-masked]>
Subject: Re: memory bandwidth problems

Tim Rowledge wrote:
> Sadly, recompiling vidcap.o with
> -funroll-loops makes no improvements :-( I can still only get 10fps of
> fullframe video. Poo.

Try -O3 too.  I did notice different code in one case, though the
performace was marginally better in one case, and marginally worse
in another *shrug*

> Are there any
> 'odd' things going on in copy_to_user() that would prevent this from being
> feasible? It looks like it should simply be amemcpy() loop.

copy_to_user() does a lot more work than that.  Take a look at all the
uaccess.* files.

- --
Andrew E. Mileski - Software Engineer [OS Team Leader]
Rebel.com  http://www.rebel.com/

- ---
This message processed by the Netwinder.Org news2mail gateway.

- --
This list is run automagically by Majordomo 1.94.4
Please send email questions to [email-masked]
To UNSUBSCRIBE: email [email-masked] with
"unsubscribe devel" as the body of your message.

------------------------------

End of devel-digest V2 #186
***************************

--
This list is run automagically by Majordomo 1.94.4
Please send email questions to [email-masked]
To UNSUBSCRIBE: email [email-masked] with
"unsubscribe devel-digest" as the body of your message.