Project 137

A forum to show of the stuff you did for your server.

Moderators: osRose dev team, ospRose dev team, osiRose dev team, Moderators

Re: Project 137

Postby PurpleYouko on Wed Mar 16, 2016 7:05 pm

Raven0123 wrote:
PurpleYouko wrote:Set compiler to align 4 bytes and recompiled.


What you want is the compiler to align to 1 byte. That way it has no way to overflow into the next structure. When I get home I'll post some code that you can use to get the full trace (if you haven't fixed it by then).


It was set to default, whatever that might be.
I'll try it at 1.
Right now I'm having a major clean-out of unused and useless rubbish that is cluttering the place up. Half of the structures are never used at all. It's just legacy code from others who have replaced code without pruning all the structures out.
Need to lookup information on NARose items, skills, quests?
Now featuring a newly completed skill tree for all classes
Formatting fixed for different resolutions
Image

"A Gazelle is nothing but a giraffe plotted logarithmicaly"
User avatar
PurpleYouko
Rose Guru
Rose Guru
 
Posts: 4733
Joined: Fri Aug 10, 2007 2:05 pm

Re: Project 137

Postby Raven0123 on Wed Mar 16, 2016 7:23 pm

PurpleYouko wrote:
Raven0123 wrote:
PurpleYouko wrote:Set compiler to align 4 bytes and recompiled.


What you want is the compiler to align to 1 byte. That way it has no way to overflow into the next structure. When I get home I'll post some code that you can use to get the full trace (if you haven't fixed it by then).


It was set to default, whatever that might be.
I'll try it at 1.
Right now I'm having a major clean-out of unused and useless rubbish that is cluttering the place up. Half of the structures are never used at all. It's just legacy code from others who have replaced code without pruning all the structures out.


The default is usually 4 bytes.

Also if you want to find out what is causing memory corruption, I would suggest set the "Page Heap" settings in gflags (it's built directly into the OS). When you run the program in the debugger, it will tell the debugger to stop once you encounter a buffer overrun or access to freed memory. This works in both Debug and Release builds.

EDIT: if you would like to see the doc for GFLags this is the link: https://msdn.microsoft.com/en-us/librar ... 57(v=vs.85).aspx

EDIT 2: I would also suggest telling it to only enable it for the application/directory you want to debug. This will prevent your system from dying due to the mistakes in system application code.
User avatar
Raven0123
osiRose dev
osiRose dev
 
Posts: 379
Joined: Tue Sep 11, 2007 11:06 pm
Location: USA, NJ

Re: Project 137

Postby PurpleYouko on Wed Mar 16, 2016 8:10 pm

It seems that setting it to 1 byte alignment completely kills all vectors.
The code doesn't even reach the part where it previously crashed. It dies the first time it encounters a vector and tries to read from it. Looking through the character data in memory shows that all vectors are impossible to access.
Only way I can even get them to register at all is to leave it set to default.
Need to lookup information on NARose items, skills, quests?
Now featuring a newly completed skill tree for all classes
Formatting fixed for different resolutions
Image

"A Gazelle is nothing but a giraffe plotted logarithmicaly"
User avatar
PurpleYouko
Rose Guru
Rose Guru
 
Posts: 4733
Joined: Fri Aug 10, 2007 2:05 pm

Re: Project 137

Postby Raven0123 on Wed Mar 16, 2016 8:35 pm

PurpleYouko wrote:It seems that setting it to 1 byte alignment completely kills all vectors.
The code doesn't even reach the part where it previously crashed. It dies the first time it encounters a vector and tries to read from it. Looking through the character data in memory shows that all vectors are impossible to access.
Only way I can even get them to register at all is to leave it set to default.


you only want to set the one data structure to align to 1 byte not everything. The macros I posted before will allow you to do that.
User avatar
Raven0123
osiRose dev
osiRose dev
 
Posts: 379
Joined: Tue Sep 11, 2007 11:06 pm
Location: USA, NJ

Re: Project 137

Postby L3nn0x on Wed Mar 16, 2016 9:02 pm

This sounds a lot like a memory management issue, not an align issue.

It crashes when you try to allocate new memory so that means that you have written outside of our allocated memory.

The memory allocated with new/malloc is defined like this :

-------BEGIN BLOCK---------
Size
Owner
--------BEGIN DATA---------- <- the pointer you get is pointing here
.. your data
---------END DATA----------
--------BEGIN BLOCK-------
...

So that means that if you write outside of you allocated block then the heap will get messed up cause the OS relies on the the size written in here to know where to allocate the next bloc of memory.

So you're writting outside an alocated block somewhere in your code.
To help you try using valgrind or https://en.wikipedia.org/wiki/Buffer_overflow_protection maybe ?
L3nn0x
osiRose dev
osiRose dev
 
Posts: 111
Joined: Wed Oct 21, 2015 8:22 pm

Re: Project 137

Postby Raven0123 on Wed Mar 16, 2016 10:38 pm

TLDR: the compiler likes to add padding to structures so they sit on 4 byte bounds. If the compiler does this, copied data may shift by a few bytes destroying pointers.

L3nn0x wrote:This sounds a lot like a memory management issue, not an align issue.

It crashes when you try to allocate new memory so that means that you have written outside of our allocated memory.

The memory allocated with new/malloc is defined like this :

-------BEGIN BLOCK---------
Size
Owner
--------BEGIN DATA---------- <- the pointer you get is pointing here
.. your data
---------END DATA----------
--------BEGIN BLOCK-------
...

So that means that if you write outside of you allocated block then the heap will get messed up cause the OS relies on the the size written in here to know where to allocate the next bloc of memory.

So you're writting outside an alocated block somewhere in your code.
To help you try using valgrind or https://en.wikipedia.org/wiki/Buffer_overflow_protection maybe ?


He should not use valgrind if hes on windows. Using gflags is the way to go because its built directly into the OS. I said alignment issues because I've seen where if you copy a data structure that is misaligned, the compiler will try to copy more bytes then you actually have aloc'd.

For example:
  1.  
  2. struct badAlign {
  3. char data1;
  4. int data2;
  5. short data3;
  6. int data4;
  7. long long data5;
  8. }

The structure above isn't aligned to a 4 byte boundary. so it may or may not put padding into the memory to align it. So the following may happen:
  1.  
  2. struct badAlign {
  3. char data1;
  4. char pad[3];
  5. int data2;
  6. short data3;
  7. char pad[2];
  8. int data4;
  9. long long data5;
  10. }


When it goes to copy this data, the size of the struct is now 5 bytes bigger then you expect it to be. I've seen cases where since it does this behavior, when copying data the data will get shifted by a few bytes every time a copy is invoked.

To align this structure better you would need to do this:

  1.  
  2. struct goodAlign {
  3. int data2; // 4 bytes
  4. int data4; // 4 bytes
  5. long long data5; // 8 bytes
  6. short data3; // 2 bytes
  7. char data1; // 1 byte
  8. }


The compiler will still fix this and go:

  1.  
  2. struct goodAlign {
  3. int data2; // 4 bytes
  4. int data4; // 4 bytes
  5. long long data5; // 8 bytes
  6. short data3; // 2 bytes
  7. char data1; // 1 byte
  8. char pad1;
  9. }


in this case the compiler only added one byte to the size of the structure.
User avatar
Raven0123
osiRose dev
osiRose dev
 
Posts: 379
Joined: Tue Sep 11, 2007 11:06 pm
Location: USA, NJ

Re: Project 137

Postby L3nn0x on Thu Mar 17, 2016 12:55 am

Yes the compiler always tries to align the struct members to gain speed at the expense of more memory. But the alignment is only needed when doing a memcpy from a buffer to the struct (such as the first sector on a hard drive or a file) because otherwise it'll be the padding that is actually filled instead of the variables you wanted. But sizeof uses the size of the struct after aligning (if it was) so this shouldn't be a problem...

I think it may be related to those static arrays you're filling.

Otherwise, why don't you use iterators instead of int in your for loop ? It'd be safer and faster to use.
L3nn0x
osiRose dev
osiRose dev
 
Posts: 111
Joined: Wed Oct 21, 2015 8:22 pm

Re: Project 137

Postby Raven0123 on Thu Mar 17, 2016 12:59 am

L3nn0x wrote:Yes the compiler always tries to align the struct members to gain speed at the expense of more memory. But the alignment is only needed when doing a memcpy from a buffer to the struct (such as the first sector on a hard drive or a file) because otherwise it'll be the padding that is actually filled instead of the variables you wanted. But sizeof uses the size of the struct after aligning (if it was) so this shouldn't be a problem...


I've had the issue before where a memcpy would copy correctly but any other type of copy wouldn't (even manually defining a copy constructor and =op)
User avatar
Raven0123
osiRose dev
osiRose dev
 
Posts: 379
Joined: Tue Sep 11, 2007 11:06 pm
Location: USA, NJ

Re: Project 137

Postby L3nn0x on Thu Mar 17, 2016 1:10 am

Lol that should be the other way around actually :?

Anyway the pack macro is really only needed when dealing with raw buffer that you want to cast into a struct. Otherwise the best thing is just to let the compiler decide cause he's much smarter than us :mrgreen:

I've seen this kind of error twice and it's really when you're messing around with raw pointers and uncontrolled buffer writes. And it's a real pain to debug because you have to check every new/malloc/realloc/delete/free and manually keep track of where and how much you write to those buffers... I'm thinking about raw heap allocated arrays mainly.

One last thing that hit me when reading your messages again Purple is that the best list container is std::list and not std::vector. And iterators are the perfect way to iterate over lists.
L3nn0x
osiRose dev
osiRose dev
 
Posts: 111
Joined: Wed Oct 21, 2015 8:22 pm

Re: Project 137

Postby Hazuki on Thu Mar 17, 2016 12:50 pm

Been a while since I've poked around Rose, good to see you still working on stuff Purple! Hope all's well :)
Image
Hazuki
Smoulie
Smoulie
 
Posts: 62
Joined: Wed Aug 08, 2007 10:09 pm

PreviousNext

Return to Pimp My Rose

Who is online

Users browsing this forum: No registered users and 4 guests

cron