Persistence - what would you use it or need it for?

Discussion in 'Plugin Development' started by EvilSeph, Feb 14, 2011.

Thread Status:
Not open for further replies.
  1. Offline

    Afforess

    Posting here because a lot of the people who made persistence happen posted in here.

    First off, I'm curious as to why there was no official announcement to dev's that bukkit added persistence support (in the form of SQLite with Ebean frontend). It's a rather big deal. There are no official explanations on how to use it (we are just assumed to completely understand a rather large, foreign, and worst, not very widely used API immediately.). I'm also confused at how Ebean was even chosen. Was there any internal discussion? Were pro/cons weighed between other, more popular alternatives? Did @Dinnerbone re-label a Russian roulette and spin that to chose? (I kid...)

    Additionally, I've noticed several quirks using Ebean.

    The most obvious issue is that it tends to spit out vague, and useless logs to the console that frighten users

    Example:
    Show Spoiler

    Code:
    DataSourcePool [MinecartManiaCore] autoCommit[false] transIsolation[SERIALIZABLE] min[2] max[20]
    SubClassFactory parent ClassLoader [org.bukkit.plugin.java.PluginClassLoader]
    Entities enhanced[0] subclassed[2]
    DataSourcePool [MinecartManiaSignCommands] autoCommit[false] transIsolation[SERIALIZABLE] min[2] max[20]
    SubClassFactory parent ClassLoader [org.bukkit.plugin.java.PluginClassLoader]
    Entities enhanced[0] subclassed[2]
    SubClassFactory parent ClassLoader [org.bukkit.plugin.java.PluginClassLoader]
    Entities enhanced[0] subclassed[2]
    


    There seems to be no way to turn that off. In fact, the logs appear even before the initial onLoad or onEnable methods are called.

    More annoying than that, there is no way to configure where the database files are created. Bukkit blindly assmes I'm using the default data folder (I'm not) and creates that and sticks it in there. This clutters the plugins folder even more.

    Worse, there seems to be no way to change the initial database once you've created it. Adding a new field, or a new table entirely both throw exceptions, and probably would require me to use SQL to modify the database by hand. If I'm going to do that, why should I bother with Ebean?

    Worst of all, EBean chokes on /reload about 35% of the time, with extremely useless exception logs that completely break the plugin. The exception seems to have to do with the way classes are identified and loaded, but that's the most I can tell you. Telling users to completely stop and start the server again is a non-starter.

    Example:
    http://pastebin.com/Gwpt8Tua

    In my experience, when Ebean works, it works perfectly. When it doesn't, it explodes into pieces.
     
    1337 likes this.
  2. Offline

    Sammy

    @Afforess I share the same frustrations, and you can add that eBean doesn't let you do @OneToMany & @ManyToOne annotations. Here is the bug report (That bug has been there forever but the post is kinda' new)
    And a user found another problem just this week, EbeanSever.save(Collection) crashes when saving a good amout of data Here is the thread
    In my opinion Ebean is an awful persistence layer =X
     
  3. Offline

    Valrix

    Which is why I'm still using my own key-value method of saving anything because so far it's working perfect, is fast, and doesn't over-complicate things. I was thinking of trying out EBean, but once I saw the commands, I said screw it.
     
  4. Offline

    alta189

  5. Offline

    Sammy

    That's kinda of walking backwards isn't it ?so long everyone asked for persistence, and now with need to nag developers to make it better not stop using it ! imho of course

    And using MySQL makes it necessary for the users to add a database server (many don't know how to do it or have the server to do it), SQLite you need to add the JAR, and this might seem silly but users are lazy as hell ^^
     
  6. Offline

    alta189

    I wrote a MySQL/SQLite library that is in Resources that you do not need any external connectors.
     
  7. Offline

    Sammy

    Well that I didn't knew, but still... I don't like using other developers code, look at the NPC library, the guy stop developing for minecraft and now we need to update a code made by another user (it's damn confusing).
    I strongly think that this kind of things must be done by the bukkit's team.

    The truth is, I'm not developing new plugins (I may never make a new one), I'm just updating (not even that until July) .
     
  8. Offline

    alta189

    I understand where you are coming from, however my Library will not break due to minecraft updating, it has no bukkit/minecraft related code!
     
  9. Offline

    4am

    I never understood this; can't you encapsulate the JAR in your plugin? Like a JAR within a JAR? I mean, it makes for multiple versions of the same library which is a bad thing, but plugins are doing this already storing it in the root folder or the lib/ directory...
     
  10. Offline

    jonathanyc

    If you boil it down to the basics, all that is really neccessary is
    • A way to store custom data about blocks, groups of blocks, players, and groups of players
    • Standard schemas for types of custom data (a schema for data defining a block as protected, a player as having a certain permission, etc.)
    Plugin developers can then expand on these base schemas/formats to add their own features - if an extended schema feature is used well enough, it will naturally become a part of the de-facto base schema.

    This is the most natural and "evolutionary" way to go about this, IMO.
     
    Trc202 and mbsuperstar1 like this.
  11. Offline

    xpansive

    My upcoming world generator will need some form of persistence to remember values so that it can avoid discrepancies when restarting the server.
     
  12. Offline

    LennardF1989

    @xpansive: In that case, take a look at my reimplementation of the default Bukkit Persistence Layer (link in my signature). It fixes quite some issues due to the way it is used and you can safely use SQLite to persist lists :)

    If you were already using Bukkit's persistence, don't worry, you won't have to change any logic at all, just the way the database is loaded into memory (about 10 lines of code, maybe less).

    Ok, I'm done promoting my topic :p
     
  13. Offline

    xpansive

    @LennardF1989 I was encountering some issues with bukkit's persistence, so I'll check out your implementation :)
     
  14. Offline

    STALKER000

  15. Offline

    alta189

    Wrong thread, plus it needs new blocks, that will require a client side mod
     
  16. Offline

    Taylor Grimes

    would you persist everything or only parts and would you want to incrementally persist or persist on sutdown
     
  17. Offline

    mossyblog

    I'd personally prefer bukkit take on a serialize/deserialization approach in a more elegant friendly manner. At the moment I'm using YAMLWriter/Reader (OSS lib) to do my persistance as the current approach recommended imho is clumsy compared to the simplicty this offers.

    I'd prefer that each plugin developer take on a NOSQL approach to the storage given most NOSQL DB's out there don't require a huge amount of effort (ie MySQL) or Server-admin overhead. The ability to attach a DB like Cassandra to the server would be a more optimal approach given its jet fast and if you architect it right you can also enable/emplower Minecraft hosters to enable developer(s) to share from the same instance of the DB (given its quite rare that much traffic between client/server would warrant one instance of Cassandra per Bukkit install).

    Ideally though off the shelf a serialization/deserialization of any object(s) that implement ISerializable or something along those lines would be ideal. I'm personally not fussed with YML for its storage i can see its immediate benefits for small lightweight plugins but i'd like to see a more cleaner API for this if its to say?

    That being said. I found when I was a Product Manger of .NET, the moment we approached the subject of allowing devs to use an internal DB / IsolatedStorage feature it got messy and bad fast. Be careful, as the problem is not so much the right usage its blocking all the wrong usage parameters that come with it... one rogue plugin can poison the well as they say.
     
  18. Offline

    XMdead

    Coming in late and I confess to not reading the whole thread, but I would suggest that an auction-house type plugin might be an excellent test case for persistence. Such a (representative) plugin would track:

    • a player's inventory in the auction house, with the ability to add or remove at any time. These items can stay in the auction house for months.
    • a player's items currently being auctioned, keeping in mind that an auction might last days, and may either be based on a fixed sale price or be one where there is a minimum sale price and then players bid higher and higher prices. The auction would also have to have an expiration time.
    • History of items bought/sold/deposited/withdrawn by a player.
    • any fees for the auctions, whether they are a fixed fee, a percentage of the sale price, or both.
    • stats for the players, any given item, etc. for example, it might be desirable to show a chart with the low and high sale prices for an item over a month, or the values throughout a given day, maybe even number of bids for an item at different times of the day (to decide when to buy/sell)
    That's all I can think of at this time, but it shows the wide variety of information that might be stored.

    The ability to choose whether the data is stored on file or integrate with a database is critical, since databases are far more resilient than file-based data storage and above all we want to avoid data loss (just imagine the riots if a player stored all of his stuff on the auction house for a year and then it got lost/wiped).

    BTW, for database integration you should make sure to stick to basic SQL or better yet use JPA, keeping the database configuration information in property files. That way a player will be able to choose which database software to use (MySQL, PostgresSQL, etc) and to change the database server being used in an instant... should the master server break or go down for maintenance (assuming the server owner bothers to use a fallback server).
     
  19. Offline

    Etsijä

    Hello to all persistence fans. I am one of the few programmers who try to keep Afforess's excellent MinecartMania plugin alive. From what I can understand, MinecartMania uses the Bukkit persistence system quite heavily, and just lately, (I guess around Minecraft 1.1 or so), the whole persistence thingy started to work quite randomly. Now you see, MinecartMania plugin is largely based on signs which are set beside/under/on top of rails, and the plugin stores and reads the information in the signs by persistence, eg.

    minecart.setDataValue("hold sign data", null);
    minecart.setDataValue("HoldForDelay", null);

    As I said, this worked before, but at some point just stopped working perfectly and started to work quite randomly. Sometimes, when I create a new sign to "program" my railway system, the plugin does take it into account and stores its info, but most of the time, it is just completely ignored.

    I am tempted to fall back into using MySQL/SQLite to get at least something to work, but before I do that, could someone hint me as to what in the way persistence is coded into Bukkit has changed lately (say, last 3 months or so)?

    Thank you in advance for your help.
     
  20. Offline

    Prgr

    I like the persistent system that HomeBukkit uses now. Once you get it down and follow all the rules, It is really useful
     
  21. IGNORE THIS IF YOU WANT, I'M WELL AWARE OF THE METADATA-SYSTEM. This was just my brain-volcano erupting. Nothing you can do about it.

    I'm just brainstorming here, I hope you see what I mean.

    Object OfflinePlayer.getProperty(String property)
    Object Player.getProperty(String property)

    boolean OfflinePlayer.hasProperty(String property)
    boolean Player.hasProperty(String property)

    void OfflinePlayer.setProperty(String property, Object value)
    void Player.setProperty(String property, Object value)

    where all properties are saved in a file and always synchronized with the file. Kinda like the Windows registry with SYSTEM.DAT. So it really doesn't matter if you request properties from OfflinePlayer or Player, and there is no difference with any cached data and saved data.

    A property could look like a permission (would have my approval):
    "pluginname.data"
    The value could be of any type. say you could store a Location, an ItemStack or a Boolean. Or String[][][]. Or Hashmap.

    If you think the plugin-like syntax would confuse devs try another separator, like ","

    Cheers!

    BUT, you know what would be really cool? Adding methods to the classes without using Reflection (and getting Spout and a lot of server owners mad at you)

    That would be really really cool.

    For example, plugins could use:
    NoCheat: int Player.getViolationLevel()
    VanishNoPacket: boolean Player.getVanished()
    WorldGuard: Region Player.getRegion()
    Multiverse-Inventories: String Player.getInventoryGroup()

    Just to show what plugin developers would be capable of. It would mean an infinite extension of extensions (yeah, right) to the existing classes.

    Why not, Java.

    I'm pretty sure this is not possible, but I dare not back that up with my knowledge on Java. Can someone confirm or deny this? (SpoutDev I'm looking at you!)

    EDIT by Moderator: merged posts, please use the edit button instead of double posting.
     
    Last edited by a moderator: May 9, 2016
Thread Status:
Not open for further replies.

Share This Page