OpenOffice.org Forum at OOoForum.orgThe OpenOffice.org Forum
 
 [Home]   [FAQ]   [Search]   [Memberlist]   [Usergroups]   [Register
 [Profile]   [Log in to check your private messages]   [Log in

Arbitrary row/column limitations?

 
Post new topic   Reply to topic    OOoForum.org Forum Index -> OpenOffice.org Calc
View previous topic :: View next topic  
Author Message
MrDoomMaster
General User
General User


Joined: 23 Apr 2007
Posts: 9

PostPosted: Mon Dec 08, 2008 12:16 pm    Post subject: Arbitrary row/column limitations? Reply with quote

Hi,

I have a serious problem with Calc's arbitrary row/column limitations. I've read a couple of threads already started on this topic but I want to express my own opinions on this matter without the clutter.

I read in one thread an argument someone gave to defend the hardcoded row/column count in Calc:
Quote:
Please remember that:

1. There has to be a limit, because no computer will ever have infinitely much memory.

2. Expanding the row limit would increase the memory usage and the size of saved files.

3. The current limit is more than sufficient for the vast majority of uses.

4. However much the row limit is increased, someone will always claim to need more. Remember that the row limit has already been increased on multiple occasions, yet the demands for more have not stopped.


1) There does NOT have to be a limit. Just because you can have unlimited rows/columns does not mean you will be using them all, and thus the user will only be using as much memory necessary to hold only the cells that actually have data.

2) As I stated in #1 in regards to memory usage, this point is not true. You obviously only write out as much data as needed and you do not reserve disk space for empty cells. For example, if the user specifies that they want a spread sheet that is 100,000 cells wide and high, yet they only have 100 cells filled with data, then you only store 100 cells in your saved file.

3) OpenOffice does not have the right to make this assumption. Only the user of the product can know what his/her requirements are. Regardless of if this product is free or not, it's the job of the programmer to make the application as extensible as possible. Any hard-coded limitations only take away from the extensibility and flexibility of the application.

4) The reason why the demand for more rows and columns exists in the first place is because of the arbitrary limitation. If the rows/columns had no upper limit then the user can create as many rows/columns as he or she desires. Whether that be 1,000,000 columns or only 30, it's up to the user to decide.

Are there any plans to remove this limitation in the future?
Back to top
View user's profile Send private message
Ed
Super User
Super User


Joined: 28 May 2003
Posts: 1041

PostPosted: Mon Dec 08, 2008 12:40 pm    Post subject: Reply with quote

For each cell used, there has to be a corresponding cell address. The number of binary digits assigned to store each cell address determines the number of cell addresses that can be assigned, and hence the number of cells that can be used. So yes, there does have to be a limit. Allowing an unlimited number of cells would require assigning an infinite number of bits to store each cell address, which is impossible.

No software designer imposes unnecessary limits just to annoy their customers. If it were possible to create a spreadsheet without any limits, don't you think that would have been done in the first place?

MrDoomMaster wrote:

Are there any plans to remove this limitation in the future?
Are you planning to do something impossible in the near future?
Back to top
View user's profile Send private message
MrDoomMaster
General User
General User


Joined: 23 Apr 2007
Posts: 9

PostPosted: Mon Dec 08, 2008 12:55 pm    Post subject: Reply with quote

Ed wrote:
For each cell used, there has to be a corresponding cell address. The number of binary digits assigned to store each cell address determines the number of cell addresses that can be assigned, and hence the number of cells that can be used. So yes, there does have to be a limit. Allowing an unlimited number of cells would require assigning an infinite number of bits to store each cell address, which is impossible.

Don't do it this way, then. Suppose you have 10 trillion cells, then you would only load the cells that you can visibly see. You may even use a memory mapped file for this. As far as calculations go, you can lazily calculate them. Calculate only the formulas that you can see and their dependencies.

Ed wrote:
No software designer imposes unnecessary limits just to annoy their customers.
Wrong. Software programmers impose unnecessary limits as cop-outs (The easy path) instead of solving the actual problem. This, in turn, ends up annoying the customers.

Ed wrote:
If it were possible to create a spreadsheet without any limits, don't you think that would have been done in the first place?

It's obviously possible to create a spreadsheet without limits, it's just harder to do and no one wants to solve that problem. That doesn't mean it isn't solvable, though.
Back to top
View user's profile Send private message
keme
Moderator
Moderator


Joined: 30 Aug 2004
Posts: 2910
Location: Egersund, Norway

PostPosted: Mon Dec 08, 2008 1:14 pm    Post subject: Reply with quote

If you want to insist that it should be possible (and desirable) to create a spreadsheet software without limitations to the sheet size, tell the developers about that.

Most users around here are not developers. In any case, the purpose of this site is advising on the use of the current software, not planning future development of it.

You need to register in the issue tracker to get the developers' attention. If you do a search there, you may find a few entries addressing this very issue. As far as I can see they're all closed, but you can request reopening if you find one that fits your needs, or you can enter a new issue report.
Back to top
View user's profile Send private message
Ed
Super User
Super User


Joined: 28 May 2003
Posts: 1041

PostPosted: Mon Dec 08, 2008 1:23 pm    Post subject: Reply with quote

MrDoomMaster wrote:

Don't do it this way, then. Suppose you have 10 trillion cells, then you would only load the cells that you can visibly see. You may even use a memory mapped file for this. As far as calculations go, you can lazily calculate them. Calculate only the formulas that you can see and their dependencies.
It doesn't matter what range of cells you "load" (whatever that means), every cell in the sheet still has to have an address.

MrDoomMaster wrote:
It's obviously possible to create a spreadsheet without limits
Is it?

Please elaborate on this "obvious" way of creating an infinite spreadsheet, which is evidently so "obvious" that no software developer in the entire history of computing has ever thought of it.
Back to top
View user's profile Send private message
Villeroy
Super User
Super User


Joined: 04 Oct 2004
Posts: 10106
Location: Germany

PostPosted: Mon Dec 08, 2008 1:53 pm    Post subject: Reply with quote

Yes, let's start another touring test with trolls who are so much smarter than all spreadsheet developers of 3 decades, but can not even compile this office suite with higher row limits. The other thread has been abandoned for 8 weeks.
Notice the nick "MrDoomMaster" Rolling Eyes
_________________
Rest in peace, oooforum.org
Get help on https://forum.openoffice.org
Back to top
View user's profile Send private message
Endolith
General User
General User


Joined: 07 Feb 2009
Posts: 15

PostPosted: Sat Feb 07, 2009 1:10 pm    Post subject: Reply with quote

Ed wrote:
For each cell used, there has to be a corresponding cell address. The number of binary digits assigned to store each cell address determines the number of cell addresses that can be assigned, and hence the number of cells that can be used. So yes, there does have to be a limit. Allowing an unlimited number of cells would require assigning an infinite number of bits to store each cell address, which is impossible.


Nonsense. There is absolutely no fundamental limit to the number of cells that could be used other than the amount of memory and disk space required to store them. The hard limitation on a specific number of cells exists because of the way the software was written. If the software were written in a different way, the limitation would not exist.

http://en.wikipedia.org/wiki/Sparse_matrix#Storing_a_sparse_matrix
Back to top
View user's profile Send private message
MrDoomMaster
General User
General User


Joined: 23 Apr 2007
Posts: 9

PostPosted: Sat Feb 07, 2009 3:01 pm    Post subject: Reply with quote

Villeroy wrote:
Yes, let's start another touring test with trolls who are so much smarter than all spreadsheet developers of 3 decades, but can not even compile this office suite with higher row limits. The other thread has been abandoned for 8 weeks.
Notice the nick "MrDoomMaster" Rolling Eyes

Wow these guys are spreadsheet developers of 3 decades and this is all they can come up with?

I don't see what my nick has to do with anything...

Endolith hit it right on the nose. OpenOffice has noticable arbitrary limitations to the normal user, which implies a really bad architectural design. I've written tile map editors for games before (similar concept, 2D array of something) and I've never had to limit the map size. It's the same concept, just different application.

This problem isn't hard to solve, and I think it's hilarious that everyone gets defensive and rude about this topic.
Back to top
View user's profile Send private message
keme
Moderator
Moderator


Joined: 30 Aug 2004
Posts: 2910
Location: Egersund, Norway

PostPosted: Sat Feb 07, 2009 4:29 pm    Post subject: Reply with quote

MrDoomMaster wrote:
Villeroy wrote:
Yes, let's start another touring test with trolls who are so much smarter than all spreadsheet developers of 3 decades, but can not even compile this office suite with higher row limits. The other thread has been abandoned for 8 weeks.
Notice the nick "MrDoomMaster" Rolling Eyes

Wow these guys are spreadsheet developers of 3 decades and this is all they can come up with?
If by "these guys" you mean OOo developers, you didn't read thoroughly. Villeroy referred to "all spreadsheet developers". Learn your history (30 years ago, a good starting point).
MrDoomMaster wrote:
I don't see what my nick has to do with anything...
No, I guess you wouldn't. Explaining it in detail would take way too muh space, and might even ruin Villeroy's Turing test, so let's just say that Doom can mean a number of things, some of which being computer related. Regardless of what meaning was intended, conceiving oneself as the master of that "Doom" says something about the person behind the nick.
Villeroy may have other reasons for calling attention to the nick, though. I don't really know for sure.
MrDoomMaster wrote:
Endolith hit it right on the nose. OpenOffice has noticable arbitrary limitations to the normal user, which implies a really bad architectural design. I've written tile map editors for games before (similar concept, 2D array of something) and I've never had to limit the map size. It's the same concept, just different application.
The link provided by Endolith describes different ways to handle sparse data, and Calc sheets are indeed stored as a condensed image of a sparse matrix (I don't know the specific structure. It's possibly a linked list, but more likely a list of self-addressing elements). What Ed referred to was not the storage space taken up by the matrix, but rather the storage space needed for the pointers addressing the cell range (be it binary pointers, numerical "RnCn" cell addresses, or "A1" type cell addresses). If the possible range is to be infinite, then the number of "digits" for direct addressing needs to be infinite too.
MrDoomMaster wrote:
This problem isn't hard to solve, and I think it's hilarious that everyone gets defensive and rude about this topic.
With all due respect, I mentioned before that you're barking up the wrong tree. While developers aren't excluded from this forum, it's primarily a venue for users to help users with the currently available features.

As for the issue at debate here: Certainly, in some instances one might wish for more freedom in layout, but all in all, those OOo applications work well, and work well together. For me, that's what defines the bottom line. There are other issues I'd sooner have the developers work on. I voted for a few of those in the issue tracker I mentioned previously.

Hope I didn't ruin that test, V. Wink
Back to top
View user's profile Send private message
Endolith
General User
General User


Joined: 07 Feb 2009
Posts: 15

PostPosted: Sat Feb 07, 2009 4:41 pm    Post subject: Reply with quote

keme wrote:
What Ed referred to was not the storage space taken up by the matrix, but rather the storage space needed for the pointers addressing the cell range (be it binary pointers, numerical "RnCn" cell addresses, or "A1" type cell addresses).


There's no reason why the number of pointers would need to be limited. The spreadsheet software should be able to hold trillions of cells, as long as there is enough disk space to store them.
Back to top
View user's profile Send private message
MrDoomMaster
General User
General User


Joined: 23 Apr 2007
Posts: 9

PostPosted: Sat Feb 07, 2009 5:52 pm    Post subject: Reply with quote

Quote:
If by "these guys" you mean OOo developers, you didn't read thoroughly. Villeroy referred to "all spreadsheet developers". Learn your history (30 years ago, a good starting point).

I don't care if this is specific to OOo or not. The fact that it isn't is sad. I don't care that hundreds of programmers haven't been able to do this over 100 years even, this doesn't mean it can't be done. Generic programming opens the door for these possibilities and if programmers can't unlock this wonderful concept and start creating applications without arbitrary limitations I can't help that.

Consider Pascal strings, they have arbitrary limitations of (in the worst case) 255 characters. However, when I use std::string I can have a string of unlimited length (The HDD and memory of my system are the limit).

Quote:
No, I guess you wouldn't. Explaining it in detail would take way too muh space, and might even ruin Villeroy's Turing test, so let's just say that Doom can mean a number of things, some of which being computer related. Regardless of what meaning was intended, conceiving oneself as the master of that "Doom" says something about the person behind the nick.
Villeroy may have other reasons for calling attention to the nick, though. I don't really know for sure.

The name "Doom" here actually relates to the game, but I don't see how this is relevant at all. Anyone who stoops so low to offend someone like this obviously knows they are wrong and cannot argue in a logical way or stay on topic.

Usually when people get desperate to win an argument they'll start getting more personal than logical, and at that point you should just accept your losses and learn from your mistakes or stand corrected.

Quote:
The link provided by Endolith describes different ways to handle sparse data, and Calc sheets are indeed stored as a condensed image of a sparse matrix (I don't know the specific structure. It's possibly a linked list, but more likely a list of self-addressing elements). What Ed referred to was not the storage space taken up by the matrix, but rather the storage space needed for the pointers addressing the cell range (be it binary pointers, numerical "RnCn" cell addresses, or "A1" type cell addresses). If the possible range is to be infinite, then the number of "digits" for direct addressing needs to be infinite too.

That's a silly argument to make. I'm assuming you're simply referring to address space. If that's the case, the application should be ambiguous to that fact. For example, if you are on a 32-bit system, you have over 4GB of memory to work with. That's 65536 rows by 65536 columns, assuming each cell takes up 1 byte of memory (Most optimistic case). However, the cells should not be limited to those dimensions. The view should expand infinitely in each direction (infinite rows, infinite columns). The only data that is saved out (and the only data that exists in memory) are those cells that actually contain information. Blank cells are simply procedurally generated. The implementation of the view can be done in several ways to prevent actual empty cells from consuming memory, and the model of the view can be done in such a way that it only serializes data that exists.

When your spreadsheet is done properly (e.g., has no arbitrary dimensional limitations), you can naturally just fill in more data into the spreadsheet on other systems with larger address spaces, such as that in a 64-bit memory space. Assuming someone types over 6GB of data in the spreadsheet, you can handle that exceptional case in any number of ways by giving the user options as to how to truncate data, move it, condense it, etc. The spreadsheet can even offer "visual guides" as to how many cells can be used on that particular system, but by no means are these LIMITATIONS, they're simply guides.

I'm not going to sit here and discuss implementation details. I have the skills necessary to implement my own spreadsheet program without any limitations, it's not impossible. And no, I'm not going to write a spreadsheet application to prove a point to you guys. If you can't just understand the logic behind it then you'll never understand it.

Quote:
With all due respect, I mentioned before that you're barking up the wrong tree. While developers aren't excluded from this forum, it's primarily a venue for users to help users with the currently available features.

I understand. At first I wasn't sure if developers viewed this forum or not. I'm not going to continue this discussion anymore since, as you pointed out, this is more oriented towards the development side and is better posted in a dev list somewhere.

I"m disabling email notifications so I will remove any temptation to review this thread in the future and post replies.

As for the issue at debate here: Certainly, in some instances one might wish for more freedom in layout, but all in all, those OOo applications work well, and work well together. For me, that's what defines the bottom line. There are other issues I'd sooner have the developers work on. I voted for a few of those in the issue tracker I mentioned previously.
Back to top
View user's profile Send private message
tekenbarba
General User
General User


Joined: 26 Dec 2008
Posts: 46

PostPosted: Sun Feb 08, 2009 12:33 am    Post subject: Reply with quote

Whilst I have no intention of getting involved in a pi**ing contest, I think that the column/row limits we see today are the result of historic decisions made when computers were far less powerful than they are today. Excel - to take a well used/known application as an example - imposed tight column and row limits until recently, it was only possible to create just over sixty five thousand rows/columns, and I believe that this was a limit imposed by the developers choice of variable type - an unsigned short in all likelihood - when the application was first created. Those limits seemed to be fine for the majority of users and have just become accepted - rather like the old Y2K issue - in my opinion. Of course I could very well be wrong about all of this.

By the way MrD - and I am asking because I am genuinely interested in the technical details of the implementation being a developer myself - how would you implement an unlimited spreadsheet package - assuming there are no machine limitations of course?
Back to top
View user's profile Send private message
Ed
Super User
Super User


Joined: 28 May 2003
Posts: 1041

PostPosted: Sun Feb 08, 2009 5:17 am    Post subject: Reply with quote

MrDoomMaster wrote:
Quote:
If by "these guys" you mean OOo developers, you didn't read thoroughly. Villeroy referred to "all spreadsheet developers". Learn your history (30 years ago, a good starting point).

I don't care if this is specific to OOo or not. The fact that it isn't is sad. I don't care that hundreds of programmers haven't been able to do this over 100 years even, this doesn't mean it can't be done.


If you think it is possible to develop a spreadsheet with infinitely many rows, you're welcome to write such a program. When you're done, can we please have a download link for it?
Back to top
View user's profile Send private message
Villeroy
Super User
Super User


Joined: 04 Oct 2004
Posts: 10106
Location: Germany

PostPosted: Sun Feb 08, 2009 11:53 am    Post subject: Reply with quote

You will never read anything creative from those trolls. All they can do is demand more junk for their veines. They hate programmers like they hate all working people.
Open source? Free software? Idiots who work for nothing!
_________________
Rest in peace, oooforum.org
Get help on https://forum.openoffice.org
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    OOoForum.org Forum Index -> OpenOffice.org Calc All times are GMT - 8 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group