Wish List and "My Research Project" Unveiled

WARNING: This post is extra extra long (even for me), the top portion consists of recent wishes, and the bottom is a very detailed account of what I've been most interested in the past few years.

As of late I've been encountering more and more tools I wish I had and less and less time to make them myself :p.

As some people may have noticed, I've been interested in GForge a bit lately, and I thought it was kinda cool that they have a SOAP web services API, and there's obvious usefulness when you see that they have a Visual Studio plugin, but I've been thinking:

These guys are an open source group with an extensible project management, tracking and collaboration system that is familiar to many developers (based on SourceForge) and pretty easy to use. It includes things like Bug Tracking, CVS/SVN support, Status tracking, Wiki, Mail lists. All of these features are really neat and usable, but I think that open source developers should be supporting the project more because there's a lot of potential, but (as SourceForge found when it stopped supporting it's open source license model) not so many people are using it's developer oriented nature to their advantage. Here are some things I would really like to see for GForge's sake:

• SharpDevelop, MonoDevelop, NetBeans IDE, and Eclipse integration.
• Can integrate SVN systems already present with bugtracker and project status id's/links.
• Support for associating projects with their forge space to allow internal IDE support for searching the project wiki, checking statuses, sending messages to project members, bug reporting.
• All of these platforms can benefit greatly from a true collaboration suite; Visual Studio 2005 Team System is great for companies that can afford to do that, but the open source and free software developers need access to these powerful tools as well.
  
• Even easier installation, currently the apt packages are pretty good and the dependency checking and such is ok, but you still have to have a lot of knowledge about how you're trying to use the system even before you've played with it. I went through several install attempts and even had some minor problems with the VM images.
• It would be pretty cool to see some sort of hybrid of Joomla/Mambo and GForge to provide standard CMS options for project websites. This would be a really nice quick-start solution for any project and if it were coupled with some nice blogging software with tag feed support on a per-user basis it could even come pre-configured to aggregate all blog entries from all users working on the project based on a unique tag for that project I think that would be just about everything a development team needs :).
  

Other than GForge, I'm sure people are aware of my interest in the Mono project. Mono is developed and maintained by a bunch of great and bright people (who's blogs I strongly recommend reading :p) and they've been making great progress. Why mention them in my wish list? Because as great as everything is going, I'd love to see even more support. The project really has the potential to put out a great cross-platform implementation of the .NET Framework but their project is not realizing it's full potential because the people working on it already have their hands full. Plenty of people are out there are playing with mono, which is great, but lets see some mobile developers get their hands a little dirtier and start making more packages for mobile platforms. What about running Mono on Windows CE or more supported packages for ARM/MIPS/SHx flavors of linux? Why use .NET CF if your embedded device has more memory and more storage space when you can benefit from the full blown features of Mono? Not to say these things aren't being looked at, and not to say the people looking at them aren't capable of doing them, but why not give some of these guys a break so they can get even more of their official projects done (Goooo WinForms! :p).

Enough with the wishes, how about the results of my actions?!

I've yet to disclose a lot of details about my personal research project. This is mostly because many of the people who read this already know what they want to and because it's somewhat complicated to explain and I haven't had the time. Well fear not those who really want to know :p, here is the information you thirst for (I presume I'm speaking to myself here, lol):

The Project: I aim to design a multi-language pervasive distributed extensibility platform for generic extensibility using dynamic methods of communication and a strong service oriented architecture.

The Research: It is my opinion that mankind has been repeating their actions in a cycle where we strive to solve inconveniences. After we solve so many of them we realize that it's too complicated now and our latest inconvenience is the number of inconvenience fixes we apply, so we then work to integrate those pieces with one another so we can use less concrete items or actions to scratch those itches. After that we find new inconveniences and start over. I would like to see if it is feasible (and if so if it is reasonable) to create a standardized system that will provide applications with standard lines of communication and service registration, recognition and consumption that can be used either standalone or as a transparent distributed system. Ideally using this system should allow us to integrate already existing technologies with new technologies without having to change either so that we can stop reinventing the wheel (which we do in way more than one way).

The Reason: As technology has been evolving the rate at which new technologies comes out is speeding up tremendously. The ultimate goal with this is to make it so that any technology can, without any prior knowledge, access the data and features of any other technology (within the system) as though the features provided were natively available in the first place. All that should need to be done to add new devices and concepts is to define new known service types, implement new service consumers on various platforms and then make the new technology compatible by exposing it's resources via a service provider. This sounds like a lot, but really it's basically what we already do. Operating systems use drivers all the time, but if this is done right we can use all of these exposed features in a distributed system.

The Method: Plenty of madness, followed by using the extensibility platform (the research project) to design and implement a personal and household device integration and automation system. The goal is to make it so that you can change the temperature of the house or turn of your kids TV while you're driving in the car touching your dash mounted screen that's communicating using the network access exposed by the bluetooth connection your car has established with your phone while getting roadside directions from the GPS location server in your phone and playing music from your wife's laptop using WiFi in the kitchen. I know this seems like a lot, but when it's broken down into small pieces it's very easy to see how everything fits together.

The Pieces: Since there are really two projects here, the pieces will be categorized as either being part of the platform or the implementation.

Platform:

• Extensibility Layer
• Allows dynamic loading of extensions in a type-safe manner with an Extension Manager that can be configured with pluggable loaders, management, and security zones to allow isolation of trusted and untrusted or critical and non-critical extensibility. This is the core of the platform.
  
• Communication Layer
• Facilitates the transport of messages (commands, requests, updates, etc) using a pluggable architecture to allow communication through dynamically defined channels in an abstract manner to allow new communication channels to be added at any time and to allow all pieces of the system to communicate using these channels without any knowledge of how they work. The communication layer should be the only section of the application that is aware of any process instances and application boundaries and it should be responsible for the detection and usage of the most optimal means of communication between providers and consumers.
  
• Distribution Layer
• Automatically provides communication for Extension Manager/Extension Manager and Extension Manager/Extension Loader across application domain boundaries using the communication layer. This should be transparent; service registration, consumption and communication should occur using a static reference to the platform host which will either be a real platform host (Extension Manager) or a virtual host (a secured proxy loaded in a sandboxed application domain).
• Configuration Layer
• Establishes a secure standard model for configuration. This should support pluggable persistence mediums, atomic synchronization through the communication layer, and configurable zones that allow various portions of the application and it's extensions to be isolated and persisted where most applicable without requiring the extensions to be aware of the location of their configuration. The configuration layer is likely to maintain a hierarchical form in which application zones are responsible for managing configuration of their children including nested application zones. Utilization of existing configuration persistence mediums is encouraged.
  
• It will be very important for there to be a rich, easy to use UI for configuration of the system as a whole, since it may actually be somewhat complicated to have various security zones communicating with one another on different machines. Also, pretty much every portion of the platform can be dynamically modified so any part of the platform must be completely configurable.
• Secure Design
• While this isn't actually an individual piece so much as a common theme it is important that all pieces of the system can be configured to be as secure or open as desired and continue all of their allowed functionality under those situations.
  

Each piece of the Platform will have to be implemented for each programming platform to be supported (ie: Java, .NET/Mono/.NET CF) but integration of some platforms may benefit more from initiatives to make languages compatible with common language structures similar to JRuby, Phalanger, IKVM, etc. SOAP, JSON and other service oriented platforms are the obvious choices for integrating the various language platforms, but I thought it would also be nice to facilitate native lines within virtual compatibility hosts (ie: host an IKVM in a .NET domain with a virtual host and then use IKVM-.NET stubs in conjunction with the native Java extensibility platform to facilitate faster translation to native communication methods using IKVM to proxy). One very important thing to make note of is that any extension can utilize the same underlying extensibility and communication frameworks to be just as extensible and distributed as the main application.

Implementation:

• Currently Planned Host Applications:
• Shell/Desktop Management Environment
• pluggable environment defining many:
• Environment providers (Windows, Linux, Mac, Windows CE, etc)
• Input providers (keyboard, joystick, remote, voice command, etc)
• Support for pluggable "Desktop Managers" to facilitate different styles of user front-end usage. A media desktop might simply expose available media, show visualizations and load semi-transparent widgets whereas somebody running the system on a touchscreen on their fridge may want something a little more home automation oriented. Either way, the fridge should technically be able to stream music while the media center turns your lights off, but the end user experience is more consistent with what they expect.
• Service Daemon
• A passive background application designed for desktop computers and servers that will load and run services in the background and allow individual platform compatible applications to serve and consume without capturing the entire environment.
• HTTP Access Platform
• Facilitates end-user service consumption through a web browser using pluggable widget/webpart style service consumers.
  
• Media Distribution Services/Consumers
• Generic service providers for exposing and streaming shared media resources (files, TV Capture output, proxied multi-cast streams, etc.).
• Generic service consumers for accessing and playing back media
• Common Device Services/Consumers
• Service providers and consumers for such things as GPS/other bluetooth devices, Printers, Scanners, IR Ports, Cameras, Microphones, etc.
  
• Automation Services/Consumers
• Provides support for registering, recognizing and exposing non-computer devices that can be controlled or polled remotely (ie X-10 household automation, RF controllable vacuums/robots).

The implementation is very incomplete in it's definition because much of the engineering thus far has been focused on the platform, but the implementation definitely needs to be solid because it is what will be used as a reference to see the capabilities of the platform. Currently I've got some good work started on the extensibility and communication layers and I've been playing with the configuration layer a bit lately too, though only on the high-level side, not so much on dynamic persistence and whatnot yet. Also, I've got a precursor to the Environment/Desktop Management Application that demonstrates a windows environment provider and a desktop provider configuration dialog with multi-head support (using display devices exposed by the environment provider) as well as simple input provider demonstration. Next is loading desktop providers and associating them with individual/multiple contiguous displays.

Anyway, this post is even longer than my average of really really long, and I'm just about out of time, but I thought I'd also draw attention to the new location of this blog :) - I'll try to let everyone know what my intentions are with it soon enough :p.

-TheXenocide

Tracking Back and Moving Forward

bunnie’s blog » Blog Archive » Scritch scritch–huh?
First, thought I'd start the day off with a trackback to some unfortunate us politics that I wish would be different. Just follow the link, read the resources and formulate your own opinion, it's pretty obvious what it says.

Also, I've just recently seen "An Inconvenient Truth" the Al Gore global warming educational film. I feel that the film is a well put together, documented and scientifically backed cry for unity the likes of which is complicated by the greedy nature of our political system and it's "campaign contributions" (That's my own personal opinion, not directly expressed in the movie). This film clearly and truthfully (look up the data) indicates the insanely critical point we're at as a species and as members (the primary unbalancing ones at that) of a planetary ecosystem. I strongly suggest you watch it because I can't do as good a job as they do at stressing how important this is. Turns out scientists have had to sue to have their names removed from the report because their claims were censored and did not want to be considered supporters of the modified message. Also, Al Gore has owned share in the carbon offsetting business for quite some time. Sorry for the hasty statements.

beyond all that, in the Software Engineering and Development side of my brain I have been planning and planning more of the development of my research project and along the way I have stumbled upon several really really neat ideas for projects I'm sure other people will be interested in too. I only wish I had the time to work on them as well, but it seems I barely have enough time to work on my own commitments. This can also be assumed as a plea for somebody... anybody, to be my hero and organize these projects into being; I'm totally game for participating in the engineering process and things of that nature but I do not have enough time to be considered a significant help on actual implementation/coding. These projects, in the long run, can also be easily adapted into my infrastructure to make a distributed pervasive system who's usefulness will extend well into the era of grid and mesh computing (provided, of course, it is maintained and the same coding concepts apply :p). Enough beckoning, to the projects!

Hah! Tricked yah (not really, but there's a little bit of background information first :p)

If you haven't checked out IKVM.Net you should. It's a Java Virtual Machine that can dynamically interpret (JIT compile) and statically compile Java bytecode to .NET CIL (supporting both Mono and Microsoft's .NET Framework). I see this project as a tremendously important idea because it would bridge the gap of the oh so similar next generation programming languages and environments currently separated by a great chasm only vaguely crossable using J#. I haven't taken the opportunity to check into support for JRuby, Groovy, JACL or Jython but I do know that it is currently aimed at support for Java 1.4. Honestly I feel this project would get some tremendous headway if it were taken seriously and contributed to by the Java/Sun and Microsoft communities. There are other projects like Phalanger that do similar managed interoperability support and I love it, but Java is the one we really need guys. There's no reason for us to constantly be reinventing the wheel on different languages and platforms when we can just bridge them together in an efficient manner and let everyone use the language that will benefit them most without making it so nobody outside their little inner circle can use it. Anyway, this rant went a little long :p, my ideas:

I feel that IKVM could use help in the following areas:
-Migration to a J2SE 5 model
-Support for bridging the existing Java/Sun written runtime instead of the GNU Classpath (this is more of a political venture) since IKVM supports JNI this should work relatively well; in retrospect another option is to bridge the awt packages and native control libraries to .NET GDI+ (System.Drawing/System.Drawing.Drawing2D) and System.Windows.Forms base classes (System.Windows.Forms.Control). Note: it seems like this is further along then i remember, which is great, but it's still incomplete.
-Support for App.config settings to support local classpath definition for applications that don't use IKVM directly.
-Attributes to apply to .NET code (ie [IKVMPackage("something.whatever")]Something.Whatever) that would allow IKVM to better translate .NET code to Java calls and to allow things like making Java stubs of classes written in .NET that don't use a cli.* package so as to be able to replace existing Java packages with native .NET ones easily. Perhaps this should include other things like the ability to rename classes and interfaces (to support different naming standards, ie: [IKVMName("SomeInterface")]public interface ISomeInterface.
--I believe that the generated property accessors should also be more like standard Java bean definitions (void setProperty(type value); type getProperty()) instead of the standard CIL set_Property and get_Property.
--MulticastDelegates (events) should function more like Java events;

Thus C# event System.AppDomain.ProcessExit is the static Java function cli.System.AppDomain.add_ProcessExit().

This should translate to
cli.System.AppDomain.addProcessExitHandler(
new ProcessExitHandler() {
void ProcessExit(Object sender, EventArgs e) { /*blah*/ }
});
or a derivative that doesn't use an anonymous inner class.

I think this would be great especially since, with full swing support and JDK 1.5 support alongside better .NET support we could fully write .NET applications using things like NetBeans IDE and even potentially add debug support, hell maybe even J2EE. The reason I consider IKVM one of the best solutions to this is that a complete implementation for interoperability requires direct communicatioon/translation between virtual machines, not just service oriented architecture (as great and important as it is). After running some simple computational tests you find that running Java code that has been statically compiled to CIL runs tremendously faster in the Microsoft.NET and Mono CLRs than it does in Sun's own JVM. Unfortunately the complications of translating from one object base to another wind up slowing some things down a little bit (Like IComparable to Comparable conversion) but still IKVM runs much faster than I could ever imagine .NET running in the JVM. This would allow Java developers to leverage and take advantage of .NET benefits without losing the cross-platform (Mono) and language specific features they love.

Other than IKVM, another great project/standard to look at is JSON which is gaining popularity. There is currently a project to support JSON-RPC for .NET called Jayrock that is pretty full featured, but I had a great idea that would be a truly impressive and usefull .NET implementation of JSON-RPC. A JSON Serialization Formatter and JSON-RPC based remoting channel would be a much more applicable approach that would make JSON totally usable with .NET applications without having to modify any code (assuming they use App.config based remoting configuration).

If anybody were to create these projects I would certainly be able to leverage the technologies in my research which I aim to be able to run services, consumers, and even desktop environments on everything from ARM mobile processors to x64 breeds simply by using a very simple base extensibility framework that is currently .NET specific but I would love to bridge support to Java with. I've even considered using db4o databases with both (if they're still language compatible) the Java and .NET libraries as a means of providing a standard consistent persistence model that requires arbitrary conversion for the storage of objects. I haven't done a full architecture review on this though, so I'm not sure what part IKVM will play in relation to db4o (ie, running Java db4o with IKVM for Java apps and running .NET db4o with native .NET or using .NET db4o for both with IKVM stubs for Java and saving Java db4o for native JVM implementations).

I'm also desperately interested in information on how well Mono runs when compiled against Linux 2.4 and 2.6 for ARM/MIPS/SH processors, including how well Mono's WinForms run on it and if there are any Mono based "Compact Framework" builds. If there aren't any Compact Framework builds it's not really that big of a problem, but a windowing toolkit for mobile devices would be much appreciated :p.

Anyway, those are my ideas/requests/plans 8) people should learn to reply sometimes, lol.

-TheXenocide