Inactive [INFO/WEB] Netstats v5.0.4 - Keeps track of player playtime and online status [RB 1.1-R3]

Discussion in 'Inactive/Unsupported Plugins' started by Valrix, Feb 13, 2011.

  1. Offline

    Valrix

    Netstats v5:
    COMPLETE!
    Keep up with development in BukkitDev!
    Netstats - Player Monitoring Plugin:
    Plugin Version: v5.0
    Web Version: v3.0

    Netstats watches players as they join and leave the server and keeps track of how long they've been logged into the server as well as keeping track of their current online/offline status while using very little system resources. The information is stored in a database which is defined by the user. First time using Netstats it will create the config file then disable itself until you change the config to match your database. Then you can reload or restart the server and it'll start collecting information immediately.
    Source Code: Netstats

    OLD CHANGELOG (open)

    Version 4.7
    • Bug fixes
    • If you don't supply a "database" in the config it will attempt to make it's own database named "NetStats" then create the table "netstats" inside it.
    • Fixed repetitive code and unnecessary MySQL connections when doing multiple queries one-after-another (reduces database calls, speeds up the code, and uses less resources).
    • Reduced code complexity (simplified certain calls and removed hacky functions to improve code performance by around 5-10% in my tests)
    Version 4.6
    • Fixed the bug that happened when trying to change the name of your stats table.
    • Fixed the bug that messed up everyone's total play time (sorry)
    • Code improvements and uses less code for database calls
    Version 4.5
    • Fixed the bug where folders weren't being made.
    Version 4.4
    • Converts the total play time to be saved as seconds instead of milliseconds to increase maximum play time. This will now match the PHP code, so if you changed it, change it back.
    • Now disables and re-enables itself instead of reloading the server when doing a wipe
    • Some code improvements
    Version 4.3
    • No longer requires the /lib/ folder since CB already contains the needed drivers
    • v5 is close to being ready and will bring large speed improvements
    Version 4.2
    • Fixes the bug that creates .stats folders instead of files.
    Version 4.1
    • Many code improvements to improve speed and memory usage.
    • Thanks to @jascotty2 for the code to fix how player and mob kills are handled
    Version 4.0
    • Many bug fixes
    • Changed how total play time was saved to the database
    • Fixed a bug with timed events
    • A new and improved Web update is nearly finished
    Version 3.9
    • Should work with RB 617
    Version 3.8
    • Fixed an issue with how files were handled.
    Version 3.7
    • Fixes issue of logged-in users being shown as offline after a reload
    • Fixed file creation bug where .stat files weren't being made properly
    Version 3.6
    • Fixed a bug where, depending on your region, your IP address was logged twice in the database. This was more of an issue with how the IP address was being grabbed than a problem with how it was stored.
    • Few other small fixes to prevent possible future bugs.
    Version 3.5
    • Changed how paths are managed by the plugin to be more OS-independent, hopefully fixing all "file not found" errors.
    • Removed checking if a database password exists or not.
    Version 3.4
    • Improved error logger that create a special log file with the exact info I'll need to find out what caused the problem.
    • Numerous bug fixes and speed tweaks
    Version 3.3
    • Added tracking of distance traveled
    • Added tracking of number of monsters a player has killed
    • Added tracking of number of other players a player has killed
    • Added the date of when the player first joined the server (after this version of Netstats has been installed, will only be correct after a full wipe of other stats)
    • Database will auto-build if it doesn't exist, otherwise it'll add columns if they're needed (so you don't have to do anything to update your database)
    • Config file now auto-updates and allows for comments (they need their own line however)
    • Many other code improvements and fixes to help it run faster and more efficiently.
    Version 3.2
    • Fixed the problem with saving to the database that everyone's been getting. (Only need the JAR)
    Version 3.1
    • Changed the namespace to com.sparkedia.valrix since I did the Java bit.
    Version 3.0
    • Rebuild of v1 to be more efficient.
    • Customized tracking based on "true" or "false"
    • What you don't want tracked won't be watched which improves performance
    • Timed updates (default is 90 seconds)
    • Fixed updating based on total actions user has done (default is 32)
    • Numerous performance tweaks
    • A "pretty" config file
    • Improved web script performance and added customized tracking (make it match your main config, variables are named the same for simplicity)
    • Web view tells when a player is still online and makes their name green if online, red if offline
    • Uses the "hybrid" method of player data saving like v1 to put less strain on the database
    • When saving to a database it is done dynamically so that only changed things are saved to cut down on the code needed and make smaller, faster calls to the database when saving (this was a big improvement on all previous versions)
    • Table name has been changed to "netstats" to not conflict with any other tables
    • NOTICE: You will need to remove your old data since it is outdated and the table is named differently with a few field name changes so make sure to use the .sql file to add it to your database. Please, before complaining about losing all the old data, realize that there's a 99% chance that it's wrong.
    Version 2.0
    • Complete revamp!
    • Now uses only flatfiles to store data, no more database issues!
    • Lots of code removed and improved for faster speed and less resource use
    • Customizable timed updates (default is every 30 seconds)
    • Customized tracking also reduces resource use by registering (or not registering) events as per your config so it only watches events that you want it to.
    • Web script will search for your CraftBukkit server when you first look at the web page.
    • You can choose where the server will save every player's .stats file by setting the "path" option in the config. The path is relative to your main server folder (where the world folder should be at). So if you set "path=../players/" then the .stats files will be in a folder called "players" in the folder that your server is in.
    • Table of player data is built depending on what your tracking and will change as you change the Netstats config file, nice and easy!
    • There are a few rules to setting paths that are listed above, make sure to read them!
    Version 1.8
    • Various code fixes and changes to improve how it runs.
    • Creates a new blank config file if it can't find one.
    • Only JAR has been changed. Get the newest one above, which I'll just reference from now on since it's faster than posting the same link over and over again.
    Version 1.7
    • Changed how files are accessed which should fix the "Too many files open" bug on Linux machines. Update the [JAR]
    Version 1.6
    • Hopefully fixed bug with connecting multiple times to a property file for each user. Only [JAR] needs updating.
    Version 1.5
    • Fixed a critical bug where user property files (where it saved your data before moving it to the database) wasn't re-defined when the plugin was reloaded. Now it looks for it after everything you do to make sure your data is always being recorded. Only the [JAR] needs to be updated.
    Version 1.4
    • Updated the files to work with the new constructor. You'll need only the [JAR] for this update.
    Version 1.3
    • Fixed a bug with how the plugin saves user data after a server crash
    • Added an Update.sql to add new columns to database table [Web.zip] (only for those upgrading from v1.2)
    Version 1.2
    • Added tracking of blocks broken
    • Added tracking of blocks placed
    • Added tracking of player deaths
    • Added more frequent tracking of total time online
    • Implemented new "safety" storage system
    • New storage system reduces number of database calls
    • Improved the way data is stored to database to take less code
    • The table of data is now ordered by who is online
    Version 1.1
    • Added tracking of player's last used IP address
    Version 1.0
    • Release of Netstats
    WEB CHANGELOG (open)

    Version 2.1
    • Some code cleanup and fixes as well as a better auto-updater that makes sure that the updater is the latest version before doing the actual update process.
    • Localization support for 7 different languages.
    • File name changes to be more netstats-specific which will prevent clashes of CSS and other pages.
    • Better include() and require() calls so a page is only added once which speeds up all the pages and keeps them from destroying each other.
    Version 2.0
    • Improved layout
    • Automatic code updates
    Version 1.0
    • Initial release of web code

    Donations: Don't forget to give amd3th some love too since he helped in making Netstats and came up with the idea (you rock!) as well as @marshall007 since he's been awesome by helping me in his spare time.
     
    Phaedrus, Niemand, D-Lew and 5 others like this.
  2. Offline

    unl33t

    @Valrix I kind of like this idea. though just as long as I can still port it into my current site without using an iframe (using wordpress with a php plugin) without having to butcher too much php code together :)
     
  3. Offline

    Valrix

    Yeah, and iframe should work perfectly fine and probably load a bit faster too since it wouldn't need to talk to the database.
     
  4. Offline

    marshall007

    Where are you concerned with optimizing speed? As I understand it, Netstats already stores the player data in flat files, and offloads the data to the database at a designated interval, or after a certain number of actions has been performed. This solution already gives the admin control over the speed because the number of db connections is indirectly configurable.

    So basically with this new solution you'd be changing the structure of those flat files and never offloading to a db? As much as I appreciate your efforts concerning the front end, I will really miss the ease of doing most of that on my own. Furthermore, as I understand it, that would give us no way to manipulate the data on our own. As an example, I was working on giving each user a profile page. The user page displayed certain 'badges' if that particular player currently had the most kills, deaths, block breaks, etc. on the server.

    In sum, I feel this solution would present more opportunity and ease of use for the average user, but for those of us that like to tinker around and/or don't have a single static page for displaying stats it's a big hit to customizability.

    Edit: I know you're working on your own user pages, achievements, and the like, but I really enjoy customizing it to my site and creating custom data in my db based on the statistics, with relative ease. Also, when redesigning, I went with Rails over a PHP framework which means integrating with your PHP frontend would be much more difficult than hooking into the db like I already have.
     
  5. Offline

    Valrix

    The great thing is the PHP would store to flat files within itself and data would be presented just the same. The only difference is not having to use a database to store it, though I guess that would mean other sites wouldn't be able to access the data, but that's somewhat the point. To ease the data collection for people who want to customize it, I would just have a function to get the player data like I would myself that can be called to grab it from the files, which are only a cache anyway. That's the only point of the DB now, is to just cache the data in case the server goes down, otherwise the data is always meant to be live and straight from the server.
     
  6. Offline

    marshall007

    It also provides universal access to the data. Even if you write methods to retrieve player data, I can only call those functions if I'm writing in PHP and am on the same server. Assuming you have proper credentials, data stored in a db is accessible from any location and on most any language (I'm mostly concerned with Ruby and Perl).
     
  7. Offline

    Valrix

    You can also get data straight from the server universally by connecting to it's port. I'll give a quick rundown of how it looks:

    Netstats <-> PHP/Ruby/Perl via a socket connection.

    That's all, in PHP it uses a quick socket connection. Here's a little code as an example:
    PHP:
    <?php
    $PORT 
    9999//the port on which we are connecting to the "remote" machine
    $HOST "localhost"//the ip of the remote machine (in this case it's the same machine)

    $sock socket_create(AF_INETSOCK_STREAM0//Creating a TCP socket
    or die("error: could not create socket\n");

    $succ socket_connect($sock$HOST$PORT//Connecting to to server using that socket
    or die("error: could not connect to host\n");

    $text "ping"//the text we want to send to the server

    socket_write($sock$text."\n"strlen($text) + 1//Writing the text to the socket
    or die("error: failed to write to socket\n");

    $reply socket_read($sock10PHP_NORMAL_READ//Reading the reply from socket
    or die("error: failed to read from socket\n");

    echo 
    $reply;
    ?>
    I've started adding the changes coming in v5 to the changelog. It will get longer since I'm not done yet. Questions, comments, and/or words of support are welcome and encouraged.

    EDIT by Moderator: merged posts, please use the edit button instead of double posting.
     
    Last edited by a moderator: May 8, 2016
  8. Offline

    marshall007

    Oh, brilliant. I must have read over where you talked about this sort of API implementation. Then yes, this is better than a database. I appreciate the discussion.

    Questions regarding this:
    1) What datatype will be returned? (JSON string please?)
    2) What sort of parameters can be through the socket (for example limit return data to one player)?
     
  9. Offline

    Valrix

    Yeah, an API is much nicer than asking a database and screwing around with all that. It's not JSON currently, I wanted to make it lean so it sends it as a string that looks somewhat like this:
    Asking for one player's data: "1:382940324:482:59:45:23:15:519:600:2:52:8:1022.58:382940324:127.0.0.1;"
    Everyone: Same as above, but each player is separated by ";"

    I made a function that'll "clean" out the data that's not needed and give the values a name in a key-value array. Here's the PHP function that'll do it:
    PHP:
    function clean($s) {
        
    $r explode(':'str_replace(';'''$s));
        
    $a['on'] = $r[0];
        
    $a['seen'] = $r[1];
        
    $a['seconds'] = $r[3];
        
    $a['minutes'] = $r[4];
        
    $a['hours'] = $r[5];
        
    $a['days'] = $r[6];
        
    $a['broken'] = $r[7];
        
    $a['placed'] = $r[8];
        
    $a['deaths'] = $r[9];
        
    $a['mobs'] = $r[10];
        
    $a['players'] = $r[11];
        
    $a['distance'] = $r[12];
        
    $a['joindate'] = $r[13];
        
    $a['ip'] = $r[14];
        return 
    $a;
    }

    $row clean(request($player));
    Oh, and I'm still working on how things will go back and forth, but once I get it fleshed out I'll post the API. This means people can be free to roll their own setup for any language now. By the sound of it, it seems like it should be fine to get rid of the database and just make a PHP-side cache that'll stay alive for 6 hours while the server is online. Once the server goes down though, it'll keep that cache until it's next successful connection to the plugin.

    EDIT by Moderator: merged posts, please use the edit button instead of double posting.
     
    Last edited by a moderator: May 8, 2016
  10. Offline

    marshall007

    Much love, this is huge. I'm a bit confused as far as the caching system goes. If a user accesses and then reloads the page and the server is online, will he retrieve the PHP-cached data generated by the first page view, or will the plugin refresh the data? I guess I just don't understand the purpose behind the six hour timeout on preserving the cache.
     
  11. Offline

    Valrix

    I've been trying to make the point of v5 being a big change since it has take a near-complete overhaul of a lot of the code to speed it up and have less chances to screw up. The way the cache works is when someone loads the page to view everyone's info, or a single player's, it'll save that to a file for no more than 6 hours as long as the server is running (I may drop that to something less depending on how often people go view the stats). This way if the server goes down and you can't fetch directly from it, when someone goes to view everyone's stats, the most recent cache it has will be shown until it can connect to the plugin again. This is just a backup to ensure stats will always be viewed, even when the server goes down, though the cached stats will obviously be a bit off, but not drastically.
     
  12. Offline

    AreLam

    I'm still following this, even though I kind of have had low hopes since the database was ripped out. Trying to think over why I want everything in a database made me realize that I never had any use for it other than displaying the data.. :p

    So anyways.. this cache file will be written and changed by php, but not (all the time) as the Minecraft server is up, am I right? I hope this is going to be a config you can change.. so there won't be many hours without cache update in case the server crashes/goes down :)

    90% done makes me happy btw!
     
  13. Offline

    Valrix

    Yeah, getting rid of the database doesn't mean loss of functionality at all.

    Correct, the cache fill will only be written to after it has "expired" which I'll include a config option so servers that go down a lot can have it backed up more often. I'm glad you're excited! It's required a lot of work to finally get it to this point.
     
  14. Offline

    Volander

    No, it's just me being up too late - I was watching the date joined on your avatar instead of the post date and though this tread had been dead since 4th Jan..

    Time to wake up.. sorry about that.
     
  15. Offline

    Valrix

    Haha, it's fine. Also, grats on being the 1000th post in my thread :p

    Ironically enough, this is also my 1000th post.
     
  16. Offline

    ibigpapa

    Was thinking that a player board in game would be cool. A huge billboard style player list with stats in game. Maybe link a button so you can cycle through the players to see stats. Or a top player bill board for different items.

    Billboard can be anything from multiple signs to maybe Wool blocks of different colors in a 20 high, 40 long square wall. Changing colors to make the words.
     
  17. Offline

    Valrix

    Hmm, what could be done is another plugin could tap into the Netstats server and do all that so Netstats won't get cluttered. Once Netstats is release the API will allow for other Plugins and web interfaces to easily get all the player's stats. Maybe go suggest it once v5 is released.
     
  18. Offline

    Niikkoo

    So i have no coding skills what-so-ever, but! I do have a request/idea

    On our server people join as guest they are then ranked up to member by joining the forum, and from member to regular when they played "regularly" for 2 weeks (say log in once every 2 or 3 days)

    Now if you could make this automatic feature it would be awesome! Maybe add easily-customizable parameters for time needed

    Btw our forum is milg8minecraft.freeforums.org if it's any help to you
     
  19. Offline

    Valrix

    This really doesn't have anything to do with Netstats, that would be something else entirely.
     
  20. Offline

    BBFPaco

    Great to see you're doing so much to update this plugin. It's one plugin I'm really excited to implement! :)

    Since you asked a page back, here are my thoughts:

    Database:

    Wouldn't a database still make sense for displaying the data on a website? Isn't it faster to query the data from a database like mysql rather than a flat file, especially if you had thousands of entries?

    It is very possible to have hundreds of users being tracked on one server and it just seems that if you want to expand on how or what data is displayed on a website it would make sense to query a database.

    Keep in mind that I don't know much about SQL at all, only a bit about web hosting.......but I have used various stats mods for different games over the years. Many of them allowed for sorting data in different ways, comparing data between users, doing some calculations to present data in interesting ways, etc.

    Tracking time:

    If I read that correctly, you meant we would see something like 219 days, instead of 7 months, 9 days?

    One of the main reasons we really hope to use this mod is to easily view how long members have been playing, last login time, etc.

    Is there some sort of PHP function out there to make calculating this easy? If not, I guess we can go off of days.

    Thank you and keep up the good work!
     
  21. Offline

    Niikkoo

    Well is it something you would be able to do?
     
  22. Offline

    Ratchet

    what kind of updates are we going to see in v5? teaser changelog? :D
     
  23. Offline

    Volander

    Try and look at the plugin named Albus, I think it can do that.

    - Volander
     
  24. Offline

    Valrix

    Actually, databases are very slow and only truly helpful when you have relationships between data tables. However, this only has one "table" of data to manage and having hundreds of pings to the same database for each person is much, much slower than directly connecting to the server or grabbing from flat files. Flat files are often called a cache and a cache is often used due to how fast you can read from them. The only thing slowing down the read/write is how fast the language can do it and the amount of RAM the server has as well as other hardware specs. If those are all top-notch, you'd have to do many times more reads to flat files than calls to a database to slow it down. The issue is db calls often block and queries have to stack, but a file read is done in nanoseconds.

    Yep, see the changelog. I've started adding teaser changes to Version 5.0 in the log.

    EDIT by Moderator: merged posts, please use the edit button instead of double posting.
     
    Last edited by a moderator: May 8, 2016
  25. Offline

    Ratchet

    will the current database entry names remain the same etc for backwards compatability?

    thanks
     
  26. Offline

    Trumple

    Hello, this looks like a great plugin but I don't understand how to use it. Is the info stored locally by the plugin in an SQL file? If so, what is the "host=" doing in the config?
    I have a MySQL server with a known hostname, with users and an empty database. Can I use that?

    Also, what must I configure in the web side of it? I see host= and user= etc. but of which database?

    thanks for any help! A brief outline of what to do would be much appreciated! :D
    Thanks for this great plugin!

    Edit: actually I'm totally confused. I have a minecraft server at the IP we'll call mc.server:1111
    I have a webhost at the ip webhost.com
    i have a MySql server at the same address as the webhost
    I don't know what 'host' my configs should point to!
    As far as I can see, there are two configs: one on the webhost called netstatus_config.php
    and one on the minecraft server called config. Could you please tell me what points to what? Or, reading about, I see you have removed databases, what can we do now?

    Thanks for clearing this up!

    EDIT by Moderator: merged posts, please use the edit button instead of double posting.
     
    Last edited by a moderator: May 8, 2016
  27. Offline

    AreLam

    The author of this plugin has been working on removing the MySQL database thing in the upcoming "v5", you could always wait until he releases that version that (hopefully) fixes the bugs with the playtime, or decide to take the slightly outdated version for a spin.

    Currently you have to specify a MySQL server (the host) with the IP/address for the plugin to store in. The web side of the plugin can then (e.g.: on another server) call this data from the chosen MySQL server. As you have the MySQL server on the same server as the website, the config for the web config would most likely be "localhost". The Minecraft server (bukkit) plugin would then have to specify the webserver in the host field (with the appropriate username/password/permission) as in something like "webhost.com" with the port your MySQL server is configured to listen to (defaults to "3306"), so the result would look like "host=webhost.com:3306" on the bukkit plugin.
     
    Trumple likes this.
  28. Offline

    Valrix

    It'll stop using the database for storage and instead rely on flat files. On the first run it'll ping a PHP script that will pull everything from the database, change it into the proper format for the server, then send it to Netstats which will save the information. After that it'll stop talking to the database and instead rely just on the plugin and web interface that'll contain a small cache for when the server is down or crashes.

    To answer the question simply, no. The new names and old ones aren't the same and new things have been added and some old removed. That's why I will have a conversion script to scrape the database of all player data the first time it runs.
     
  29. Offline

    marshall007

    Databases scale incredibly well, you are correct. However, they are designed to store relatively persistent information that your application can read, manipulate, and then return unique results. Notice that the data manipulation is occurring in the application layer, not in the database. If you get into a situation where you're having to update table entries on a very regular basis, the infrastructure is no longer advantageous. This is a distinction I'm happy to see Valrix make.

    I assume his implementation probably isn't reading data from the disk except upon initialization. Once he has all this data in memory, there'd be no reason to. He can then manipulate the data as players interact with the environment in real time, and return those values whenever they are requested. Furthermore, whenever he decides to write the data to file, he doesn't have to parse anything. He can simply overwrite the entire file with the new data.

    Basically, all the data is stored in memory. If my assumptions are generally correct, we should see much faster response times from v5 than we do currently.
     
  30. Offline

    Valrix

    v5 isn't ready yet, but yes it will no longer contain databases natively. People can feel free to alter the code to work with a database, but flat file caching is much faster anyhow and is only needed when the server isn't running. The config will be more simplified too. All I can really say is you'll need to wait until the update is ready, that's when I'll post everything that people need to know on how to install and configure it as well as the different commands and API format.

    That's correct. In v5, once enabled it starts its own server and if any players are online, it loads their very small stat files into memory matched to their UUID. Whenever a player does something it quickly changes the loaded values, and if many are being saved, it changes them all first before saving to speed up the write.

    EDIT by Moderator: merged posts, please use the edit button instead of double posting.
     
    Last edited by a moderator: May 8, 2016
    Trumple likes this.
  31. Offline

    Trumple

    Thanks for the explanation! I missed the port out and that caused the server to fail to load up (it would basically hang at startup and therefore prevent logins)
    I will try adding the port and see if thats helps, I'll also get to configuring the webhost side too
    Thanks again
     

Share This Page