Project 137

A forum to show of the stuff you did for your server.

Moderators: osRose dev team, ospRose dev team, osiRose dev team, Moderators

Re: Project 137

Postby rl2171 on Mon Mar 14, 2016 8:46 pm

Raven0123 wrote:
rl2171 wrote:Very nice Purple!!

Can't wait to help with the testing, sounds like it will be alot of fun again!


o.o YOUR ALIVE!!!! Welcome back! how have you been?


I have been alive and never really away from here, I pop in and out to see what is going on from time to time. I don't think I have posted anything in quite some time.

Well, I have been great, will be getting married in August with a great lady, so a years worth of planning is getting closer and closer, then of course the 10 day honeymoon afterwards.

Here and there I will put up ktrose or osirose up on my computer and "play" for awhile, then take it down. Look forward to testing something new once again, it has been so long!
Image
rl2171
Admin
Admin
 
Posts: 1706
Joined: Mon Aug 06, 2007 5:17 pm
Location: Sacramento, CA USA - GMT-8

Re: Project 137

Postby aerocoke on Tue Mar 15, 2016 5:42 am

Big work ! That's some quite impressive stuff man.
Out of curiosity did you try to make the client crash ? Like with too much speed or a too big monster ?

Monster Factions. Ever wondered what would happen if there was a reputation system in which you could end up being loved by ever Jelly bean in the world or hated by Woopies because of actions you took or decision that you made? Would you like to lead an army of Golems to attack Zant? or make friends with Aquas and lead your allies in an Epic defense of Junon polis against an invading Porky army? Do the lore quests! find out the background stories! figure out why it is that the respawn points in Junon all look like the head and upturned hands of stone giants or what the deal is will all the jelly bean statues in Adventurers plains. Earn friendship or eternal enmity with the factions of monsters.


ahah I chuckled hard imagining it, that would be awesome ! It reminds me of an old rose cinematic where moldies invades junon.
There's something i've been wondering tho, do you think we can make the monsters attack the NPCs ? How exactly does rose considers the NPCs ? are they inanimate object that speaks or ?

Because for a long time i had this idea of patrolling NPCs and NPCs being attacked by some monsters but i didn't know how rose handled everything officially.
aerocoke
Jelly Bean
Jelly Bean
 
Posts: 16
Joined: Wed Feb 24, 2016 2:31 am

Re: Project 137

Postby PurpleYouko on Tue Mar 15, 2016 2:27 pm

I haven't really delved into the way that NPCs work in the client.
One thing i do know is that they have no animations associated with them.
However, a few NPCs appear to use the models of monsters so it could well be that they are handled as special case stationary monsters.
Either way I'm going to rewrite the code that handles them anyhow. I might just entirely replace the regular NPCs with VPCs at some point so that they are able to move around and pitch in to help defend a town. That way they won't be available for normal shops or quests when the town is under attack.
Need to lookup information on NARose items, skills, quests?
Now featuring a newly completed skill tree for all classes
Formatting fixed for different resolutions
Image

"A Gazelle is nothing but a giraffe plotted logarithmicaly"
User avatar
PurpleYouko
Rose Guru
Rose Guru
 
Posts: 4733
Joined: Fri Aug 10, 2007 2:05 pm

Re: Project 137

Postby Sdaak on Tue Mar 15, 2016 3:25 pm

Pretty nice list, its an impressive one.

I'm also amazed that you managed to stay independant all these years, i'm pretty sure some big servers like arua asked you to join and yet you seem to have always managed to be yourself that's awesome.
Hey hey now i want to play that teaser was a bit too good ;)
Sdaak
Jelly Bean
Jelly Bean
 
Posts: 14
Joined: Wed Mar 02, 2016 6:53 am

Re: Project 137

Postby PurpleYouko on Wed Mar 16, 2016 3:26 pm

Yesterday I had about 4 hours of free time so I figured it was time to dig in to the mechanics of crafting.
So I loaded up my servers, fired up the client and logged in with a noobie dealer, bought a few craft skills and just to make sure everything was working properly, I returned to the character select screen and relogged her.

CRASH Goes the server......
WTF I think. It worked just a couple of minutes ago. And to top it off, the debugger failed, saying "no debug symbols saved" or some such bollocks.... :(
Checked the database and all the character info is good and in the right place. the only difference between the save data no and before is that I have a few hundred more skill points and 5 shiny new skills at level 1.

So I recompiled the server, being careful to make sure that it was set the compiler to /Zi in the properties C++ section (it was already but this is c++ we are talking about)
Logged in again.
Same error but at least this time I could debug it.

This is where it threw a wobbly fit on me. VisibilityList() in PlayerFunctions.cpp
  1. VisiblePlayers.clear();
  2.     VisibleDrops.clear();
  3.     VisibleMonsters.clear();
  4.     VisibleNPCs.clear();
  5.     VisiblePlayers = newVisiblePlayers;
  6.     VisibleDrops = newVisibleDrops;
  7.     VisibleMonsters = newVisibleMonsters;
  8.     VisibleNPCs = newVisibleNPCs;
  9.     newVisiblePlayers.clear();
  10.     newVisibleDrops.clear();
  11.     newVisibleMonsters.clear();
  12.     newVisibleNPCs.clear();

It crashed on the highlighted line with some garbled crap about a corrupted heap.
Basically it's a memory management problem
I've been running into this kind of thing a lot lately.
A couple of weeks ago I was getting random client crashes immediately after login which I managed to trace back to corrupted skills data being sent to the client.
Skills were loading in from the database perfectly well and then they were getting messed up during the loading of inventory items. Specifically the first 3 skills were being completely deleted when the code loaded the item 'stats' value. There was no code that could possibly have been writing to the skills directly so I concluded that the memory location in which the skills were held was being written to by the item loading code.
But HOW?
In Plyer.h the inventory was declared CItem items[MAX_INVENTORY]; prior to the declaration of the skills SKILLS cskills[MAX_ALL_SKILL]; and then later the skills were loaded first, followed immediately by the inventory items
I tried everything i could think of.
I added specific constructors to the CItems and cskills structs (yes they are structs rather than classes but that shouldn't make any difference). It didn't help.
Eventually I sidestepped the underlying issue by moving the skills declaration to BEFORE the inventory declaration. It worked fine after that but there is still an underlying problem that I haven't been able to identify so I'm not really happy with this kind of band-aid fix.

Anyway, back to the issue that I have now.
The code in the VisibilityList function is all kinds of UUUGGHHHH. I hate it but as yet I don't see any better way to write it (and that sucks)
I hate that it makes temporary vectors and then scans through the PlayerList, MonsterList, NPCList and the DropsList one by one and compares every single member of those lists against the existing list of the same type (again by scanning it item by item) and then checking the distance to said item against MINVISUALRANGE before building the temporary vectors from the ground.... on every cycle!
before finally wiping the old lists and copying the new ones into them.
That is a horrible mis-use of processor time IMO, but HOW to do it better is the question?
And more to the point WHY is it suddenly starting to crash now when this same code has worked flawlessly for years?

I'm going to have to re-code the whole frickin server at this rate with memory management as the primary concern.... What a horrible thought...

Also WHY is the NPCList made up of a vector of CNPC* objects rather than a vector of shorts containing the clientids of the NPCs in visual range. That would take up so much less memory and it would save having to do a cast for every single one in the map on every cycle to compare it to the one in the list
And why use vectors at all if we are only adding items to the end of the list and never erasing one from the middle?
Vectors require a block of contiguous memory so they grab HUGE chunks of memory to reserve for filling up later.
A List or even a dynamic array would both work in smaller chunks of memory, just grabbing a chunk big enough for the next entry and saving a pointer to the new location automatically. (an automatically managed link list for those who come from a C background)

Today I have a few hours ahead of me where (as of right now) nothing here is broken and needs to be fixed yesterday, no coding (for work) needs to be done and no students, faculty, staff or professors need to be taught how to use a mass spectrometer, remote manipulator arm or laser ablation system. Basically I have a completely free day, pending any disasters that might occur later. :D
So I'm gonna figure a way to completely rewrite all the visibility process so that they work in a much more effective manner (and hopefully don't crash any more)

Please feel free to chime in with any comments regarding memory management, vectors, arrays, lists or classes. I'm a little rusty in these areas so I'm on a research and development mission. :D
Need to lookup information on NARose items, skills, quests?
Now featuring a newly completed skill tree for all classes
Formatting fixed for different resolutions
Image

"A Gazelle is nothing but a giraffe plotted logarithmicaly"
User avatar
PurpleYouko
Rose Guru
Rose Guru
 
Posts: 4733
Joined: Fri Aug 10, 2007 2:05 pm

Re: Project 137

Postby Raven0123 on Wed Mar 16, 2016 4:32 pm

PurpleYouko wrote:Yesterday I had about 4 hours of free time so I figured it was time to dig in to the mechanics of crafting.
So I loaded up my servers, fired up the client and logged in with a noobie dealer, bought a few craft skills and just to make sure everything was working properly, I returned to the character select screen and relogged her.

CRASH Goes the server......
WTF I think. It worked just a couple of minutes ago. And to top it off, the debugger failed, saying "no debug symbols saved" or some such bollocks.... :(
Checked the database and all the character info is good and in the right place. the only difference between the save data no and before is that I have a few hundred more skill points and 5 shiny new skills at level 1.

So I recompiled the server, being careful to make sure that it was set the compiler to /Zi in the properties C++ section (it was already but this is c++ we are talking about)
Logged in again.
Same error but at least this time I could debug it.

This is where it threw a wobbly fit on me. VisibilityList() in PlayerFunctions.cpp
  1. VisiblePlayers.clear();
  2.     VisibleDrops.clear();
  3.     VisibleMonsters.clear();
  4.     VisibleNPCs.clear();
  5.     VisiblePlayers = newVisiblePlayers;
  6.     VisibleDrops = newVisibleDrops;
  7.     VisibleMonsters = newVisibleMonsters;
  8.     VisibleNPCs = newVisibleNPCs;
  9.     newVisiblePlayers.clear();
  10.     newVisibleDrops.clear();
  11.     newVisibleMonsters.clear();
  12.     newVisibleNPCs.clear();

It crashed on the highlighted line with some garbled crap about a corrupted heap.
Basically it's a memory management problem
I've been running into this kind of thing a lot lately.
A couple of weeks ago I was getting random client crashes immediately after login which I managed to trace back to corrupted skills data being sent to the client.
Skills were loading in from the database perfectly well and then they were getting messed up during the loading of inventory items. Specifically the first 3 skills were being completely deleted when the code loaded the item 'stats' value. There was no code that could possibly have been writing to the skills directly so I concluded that the memory location in which the skills were held was being written to by the item loading code.
But HOW?
In Plyer.h the inventory was declared CItem items[MAX_INVENTORY]; prior to the declaration of the skills SKILLS cskills[MAX_ALL_SKILL]; and then later the skills were loaded first, followed immediately by the inventory items
I tried everything i could think of.
I added specific constructors to the CItems and cskills structs (yes they are structs rather than classes but that shouldn't make any difference). It didn't help.
Eventually I sidestepped the underlying issue by moving the skills declaration to BEFORE the inventory declaration. It worked fine after that but there is still an underlying problem that I haven't been able to identify so I'm not really happy with this kind of band-aid fix.


That sounds like a memory alignment issue. Did you check to see that the structure is 4 byte aligned? If it's not try telling the compiler to not use alignment on the structure. I could tell you if that is the issue if I could see the code.

The following define will tell the compiler to ignore alignment issues.
  1. #ifdef _WIN32
  2. #define PACK(...) __pragma(pack(push, 1)) __VA_ARGS__ __pragma(pack(pop))
  3. #else
  4. #define PACK(...) __VA_ARGS__ __attribute__((__packed__))
  5. #endif


If you need an example, its used here: https://github.com/RavenX8/osIROSE-new/ ... ckettype.h
User avatar
Raven0123
osiRose dev
osiRose dev
 
Posts: 379
Joined: Tue Sep 11, 2007 11:06 pm
Location: USA, NJ

Re: Project 137

Postby PurpleYouko on Wed Mar 16, 2016 5:03 pm

It seems like Vectors probably are the best kind of list to use so I found a much more efficient way to actually use them.

old code
  1.  
  2. std::vector<CNPC*>          newVisibleNPCs;
  3. for(unsigned i=0; i<map->NPCList.size(); i++)
  4.     {
  5.         CNPC* thisnpc = map->NPCList.at(i);
  6.         float distance = GServer->distance( this->Position->current, thisnpc->pos );
  7.         if ( GServer->IsVisible( this, thisnpc ) )
  8.         {
  9.             if ( distance < MAXVISUALRANGE )
  10.             {
  11.                 newVisibleNPCs.push_back( thisnpc );
  12.             }
  13.             else
  14.             {
  15.                
  16.                 this->ClearObject( thisnpc->clientid );
  17.             }
  18.         }
  19.         else
  20.         {
  21.             if ( distance < MINVISUALRANGE )
  22.             {
  23.                 newVisibleNPCs.push_back( thisnpc );
  24.  
  25.                 //LMA: WarpGate.
  26.                 if(thisnpc->npctype>10000&&(thisnpc->npctype==GServer->WarpGate.virtualNpctypeID))
  27.                 {
  28.                     GServer->pakSpawnIfoObject( this, GServer->WarpGate.virtualNpctypeID );
  29.                 }
  30.                 else
  31.                 {
  32.                     GServer->pakSpawnNPC( this, thisnpc );
  33.                 }
  34.  
  35.             }
  36.         }
  37.     }
  38. VisibleNPCs.clear();
  39. VisibleNPCs = newVisibleNPCs;
  40. newVisibleNPCs.clear()
  41.  

Declares a local instanced Vector.
Scans NPC list in the map
Casts a CNPC object
Calls GServer->IsVisible( this, thisnpc ) on it (scans the entire VisibleNPCs (the old list) list and returns true if it finds thisnpc. Else returns false)
Checks distance to thisnpc.
Spawns/despawns thisnpc on the client
Adds thisnpc to the instanced vector if it's in range
Does some other weird and pointless stuff with warp gates (Assumed to be pointless for now anyway. Will find out after logging in ;) )
Clears VisibleNPCs
Copies instanced local vector back into VisibleNPCs
clears instanced local vector

New version
  1.  
  2. for(unsigned i=0; i<map->NPCList.size(); i++)
  3.     {
  4.         CNPC* thisnpc = map->NPCList.at(i);
  5.         float distance = GServer->distance( this->Position->current, thisnpc->pos );
  6.        
  7.         if ( GServer->IsVisible( this, thisnpc ) )          //NPC is currently in the list
  8.         {
  9.             if( distance > MAXVISUALRANGE )                 //Not in visual range
  10.             {
  11.                 VisibleNPCs.erase(VisibleNPCs.begin() + i); //Not visible any more so delete the NPC from the list
  12.                 this->ClearObject( thisnpc->clientid );     //clear it in the client
  13.             }
  14.         }
  15.         else    //not currently in the list
  16.         {
  17.             if ( distance < MAXVISUALRANGE )                //Is now in visual range
  18.             {
  19.                 VisibleNPCs.push_back( thisnpc );           //Add it to the list
  20.                 GServer->pakSpawnNPC( this, thisnpc );      //Spawn it in the client
  21.             }
  22.         }
  23.     }

All actions taken on EXISTING vector
NO clearing
NO instanced local vector
NO copying data structures.

It either adds (pushback())
or removes (erase)
elements in the existing visibility list based on logical calculations of visual distance.
Still calls GServer->IsVisible( this, thisnpc ) on each NPC in the NPCList though. i can't find an easy way to get rid of that yet.

Compiles OK. About to fire it up and log in now

Also deleted a bunch of unused variables in some headers while I was tidying up ;)

Servers up with no problems

Client up and ...... Same crash again
corrupt_heap.jpg


this time it's on the line
  1. VisibleNPCs.push_back( thisnpc );           //Add it to the list

Checked all the variables in thisnpc and all are present and correct.
It would appear that the thing that is getting corrupted is vector<CNPC*> VisibleNPCs; // Visible npcs

Thanks Raven.
I'll give that a try
Need to lookup information on NARose items, skills, quests?
Now featuring a newly completed skill tree for all classes
Formatting fixed for different resolutions
Image

"A Gazelle is nothing but a giraffe plotted logarithmicaly"
User avatar
PurpleYouko
Rose Guru
Rose Guru
 
Posts: 4733
Joined: Fri Aug 10, 2007 2:05 pm

Re: Project 137

Postby PurpleYouko on Wed Mar 16, 2016 5:17 pm

tried adding that snippet of code at teh top of sockets.h where it will be loaded by everything in the server.
Compiled fine but same error in the same place.

I will look into memory alignment issues a little more and maybe try to force it to align

ABE
Set compiler to align 4 bytes and recompiled.
Exactly the same result
with same build logged in a character that has no skills. Logged in fine but after using /go 2 to transport to ZANT I could immediately see ALL the NPCs in the area (normally can't see the ones in the shops as they are too far away). After moving a few feet the server crashed again. I was unable to make much sense of where it actually crashed. The call stacks went through hundreds of lines of stuff in Windows.h :?
Reset the original visibility code AND set compiler back to default and recompiled. (I know. Changing both at once was a bad idea)
This time logged in with same character (no skills) and she runs around just fine
Logged in with a different character that has skills. Ran around just fine.
Logged in with the same infamous dealer----- CRASH

Also I get the same crash every time I exit the game or return to the char select screen now. Wasn't having that issue prior to yesterday.
And the weird thing with that bug is that I can't debug it. The debugger claims that there is no source available
Need to lookup information on NARose items, skills, quests?
Now featuring a newly completed skill tree for all classes
Formatting fixed for different resolutions
Image

"A Gazelle is nothing but a giraffe plotted logarithmicaly"
User avatar
PurpleYouko
Rose Guru
Rose Guru
 
Posts: 4733
Joined: Fri Aug 10, 2007 2:05 pm

Re: Project 137

Postby PurpleYouko on Wed Mar 16, 2016 6:14 pm

It's getting worse.........

Absolutely no changes anywhere.
Logged in with the same characters that worked 100% 10 minutes ago. Crashed the server.
Shut down and restarted everything
Logged in with the exact same characters again. crashed the server AND crashed the client. That's a first.

It's like every time I try to fix one thing in this project I get sidetracked for days and days with other weird bugs. It drives me frickin nuts.

And just to throw another spanner in the works, BOTH of the errors are not related to the previous issue in any way shape or form as far as I can see

The Client is crashing on receipt of the questdata and wishlist login packet. There's a big fat impossible to exist object in the wishlist. 3 of them in fact.

The Server is crashing when a monster moves and sends a packet to all VISIBLE Clients. So there's a link back to visibility lists but it's a different list than the one I've been working on....
Need to lookup information on NARose items, skills, quests?
Now featuring a newly completed skill tree for all classes
Formatting fixed for different resolutions
Image

"A Gazelle is nothing but a giraffe plotted logarithmicaly"
User avatar
PurpleYouko
Rose Guru
Rose Guru
 
Posts: 4733
Joined: Fri Aug 10, 2007 2:05 pm

Re: Project 137

Postby Raven0123 on Wed Mar 16, 2016 6:56 pm

PurpleYouko wrote:Set compiler to align 4 bytes and recompiled.


What you want is the compiler to align to 1 byte. That way it has no way to overflow into the next structure. When I get home I'll post some code that you can use to get the full trace (if you haven't fixed it by then).
User avatar
Raven0123
osiRose dev
osiRose dev
 
Posts: 379
Joined: Tue Sep 11, 2007 11:06 pm
Location: USA, NJ

PreviousNext

Return to Pimp My Rose

Who is online

Users browsing this forum: No registered users and 4 guests

cron