0



Firstly let me mention that I'm new to web-frameworks.

I have to write my first web-app for a Uni project. I spent two weeks learning Grails and Django. Started working with Rails yesterday and loved it. So I've decided to go with it and discard my work in the other frameworks.

About the app
It's supposed to be a Twitter app that utilizes Twitter's Streaming API to record tweets which match a set of specified filters. (I'm going to use the Tweetstream gem which takes care of connecting to Twitter and capturing matching tweets).

The app's web interface should have the following functionality -

  • Creating new requests
    The user inputs a set of filter parameters (keywords to track) & URL/username/password of an existing PostgreSQL or MySQL database.

    When a request is created, the web-app spawns a background ruby process. This process connects to Twitter via the Tweetstream gem. It also connects to the database specified by the user to stores received tweets.
  • View/terminate of existing requests
    The user should be able to see a list of requests that are running as background processes by visiting a URL such as /listRequests.
  • See further details about a process/terminate the process
    The user should be able to go to URL such as /requests/1/detail to view some details (e.g how long request has been running, number of tweets captured, etc). The user should also be able to terminate the process.

My inexperience is showing as I'm unable to comprehend -

  • what my models should be (maybe Request should be a model. Tweet doesn't need to be a model as it's not being stored locally)
  • how I'm going to connect to remote databases.
  • how I can create background processes (backgroundrb??) and associate them with request objects so that I can terminate then when the user asks.

    At the end of the day, I've got to build this myself, so I'm not asking for you to design this for me. But some pointers in the right direction would be extremely helpful and appreciated!

    Thanks!

    • infiniteloop
    • 2,112
    • 2
    • 18
    • 21
    • Oi. They assigned you this as your *first* project‽ – Matchu Nov 21 '10 at 03:10
    • Yep. This is only step one of the project. My degree doesn't actually involve much programming but I've done quite a bit of it on my own. I'm a quick learner so I think I'll be okay :) Got some of it done in Django already. Wrote a Java app to handle the database and twitter functionality. Got stumped when I had to invoke it from within Django using Python's subprocess management. – infiniteloop Nov 21 '10 at 03:15

    2 Answers2

    1

    Quite a project. Most of what will be challenging is not related to rails itself, but rather the integration with background processes. backgroundrb is a bit out of fashion. The last commit on the main github project is over a year ago, so it's likely not up to snuff for Rails 3. Search around and evaluate your options. Resque is popular, but I'm not sure if your real-time needs match with its queue-based structure.

    As for your app, I see only a single model, but don't call it request. That's a reserved name in rails. Perhaps a Search model, or something along that line.

    Connecting to different databases is straight forward but will require direct configuration of your ActiveRecord class during operation rather than using database.yml.

    jwarchol
    • 1,906
    • 15
    • 12
    1

    Hmm.

    Since the web app is just a thin wrapper around the heavy-lifting processes, it might be more appropriate to just use something like Sinatra here. Rails is a big framework that pulls in lots of stuff that you won't need for this project, even though it will work.

    Does the "background process" requirement here strictly mean a separate process, or does it just mean concurrency? TweetStream uses the EventMachine gem to handle updates as they come, which uses a separate thread for each connection. It would be quite possible to spawn the TweetStream clients from a simple Sinatra web app, keep them in a big array, have them all run concurrently with no trouble, and simply run stop on a given client when you want it to stop. No need for a database or anything.

    I'm not sure exactly what your prof is looking for you to do here, but MVC doesn't really fit. It's better to work with the requirements than to mush it into a design pattern that doesn't match it :/

    Even so, I <3 Rails! Definitely get on that when you're working primarily with objects being represented in a database :)

    Matchu
    • 83,922
    • 18
    • 153
    • 160
    • The "background process" requirement is simply that the Twitstreams keep running persistently in the background until terminated explicitly by the user. I'll look into Sinatra and see what I can do with it! – infiniteloop Nov 21 '10 at 04:23