Is this the final centralized topic for this idea now?   Regardless, let's see how far we can take this idea so that Frogboy responds.

I really like the proposed idea of being able to focus on a sub category. It makes sense from a lore perspective and is also very useful from a gameplay perspective (short on food? focus your research on it. Might not turn up exactly what you want but it'll be useful to fix your crisis all the same).

That said, it might be worth having a wildcard option that'd function somewhat similarly to what is in place right now. You tell your researchers to focus on civ based improvements but don't give them a direction - this'd give them the freedom to go all DARPA on it and come up with some of the more exotic tech - the stuff you didn't even realise you needed. This'd be an excellent place for red techs to show up, and those once-in-a-lifetime rare ones.

yeah i fully endorse this idea.  i really like the randomness and didn't want to give it up, but this way it makes more sense and is still random(its even MORE random mwahahaha!).

Research in 4x games has always bothered me.  In most games, the 'strategy' part of research is simply deciding which uber tech you want and planning how to get there most efficiently while not gimping your empire with nothing but research buildings.     

The concept of a tech tree always bothered me because you knew what you were getting.  In almost every game you knew exactly what was coming up and you knew exactly WHEN it was coming up.  To use the same example I used in that other thread, in Civ4 you knew when you'd hit rifling to the exact turn so you knew exactly when your macemen/musketeers would be upgradeable.  It just always struck me as a crummy mechanic because technology and research, especially prior to modern times, didn't work that way.  Discoveries were accidental results of studying a particular subject.  

So when I first saw Elemental's mechanic, I loved it.  After playing through the Beta (since 1g or so) I still like it.  My only issue is the immersion.  You direct your subjects to study in a particular area to see what they can figure out and they come back with... a list?  

I love the idea of having random techs available and of having the potential for major, society changing discoveries (what I envision the "red" discoveries as).  I really like that it is partially out of your control.  The Sovereign directs research on a macro level, but your researchers are the people who come up with the discoveries.  I just don't like the way it is presented to the player.  It feels too.. disjointed?

So I thought each macro category (civilization, warfare, magic, adventure, and diplomacy) could have subcategories.  You select one of those to focus your research and the game uses the already built in probabilities for each possible tech to decide which one you discover.  When you've reached a discover, your researcher brings that discovery to you.   

Certain subcategories would be available from the start and others may only be available when you've discovered whatever technology spawns that "branch" of research.  Say you're researching "weapons" under Warfare.  You discover "Archery" and learn the first weak bow weapon.  You also unlock the "ballistics" research category (under Warfare) which covers catapults, trebuchets, bow advancements, and all forms of standard ranged combat.  I'm not sure how much synergy the devs want between research trees, but there could even be requirements from other categories (research economics in civilization to unlock the "Trade" category under Diplomacy).  

I realize that the idea brings up other issues and I haven't fully thought through them yet.  

1) Does it remove too much player control?  Will players get irritated not being able to get exactly the tech they want? (This is potentially a question under the current system as well)

2) Is it even feasible given how the current system is set up to change like this in the given time?  I think I understand the devs intent with designing the tech system the way it is, but I may be off by miles.  If so, does this even conform to their goals for the tech system?

3)  How do you present existing synergies to the players?  (also an issue in the current system, if there are any).  IE, if the initial metal weapon tech under warfare requires something in addition to whatever comes right before it on the tech tree (Mining, under civilization for example) how do you let the player know that there are other tech's he needs?

4)  How do you handle the case of a player having terrible luck and the game always selecting some fancy yellow or red tech to reward them with but skipping over something green, but terribly important (farming 1)? (I think you can have a modifier that increases the chance of you discovering techs that you have vastly "outleveled)

I have plenty of other thoughts on the topic, but I'm just trying to do an initial brain dump to help the conversation before I crash for the night.  I realize there are probably many more issues my idea brings up.  

I don't want to alter the tech system drastically (thought it may seem that way).  I really, really like the design behind the system.  This whole idea started as a way to illustrate my point that, while the current tech system is great, it needs to be presented in a different way to be more a part of the game world and less an obvious game mechanic.  

If you're going have randomness representing the fact that you can't just pick a technology and say "go research that because I want longbows and that gives me longbows" then reaching a research goal and getting presented with a list seems counterintuitive.  

Okay, I think this horse is dead.  I'm going to bed now.   

I honestly like this idea especially at the beginning. But I wonder toward the late game when you really want to learn a specific tech because you know it will help you do x,y,and z and your minions keep coming back to with techs you don't want.

I love the random system, each item in the research tree would heave an ever increasing cost as you go up levels of course, you would earn your research points each turn, and the game would randomly check each time at each cost level to see if you got it, working the way up the tree.

This would stop maxing out one line in the tree to gain supremacy, and would reduce micromanagement.

So say we have 220 points of research up our selves, we have 6 available techs. T1-6.

T1 to 4 Need 220 points, T5 needs 240 points and T6 needs to 280 points. Now you could either need the minimum points and get one of the random 220 points one, or you could need the average  234 points and then get anyone of the techs with a biased towards the cheapest.

ie a 5% chance of getting the 280, a 10% chance of the 240 and 85% chance of one of the 220's.

Looking at the tree what little I have seen, it might need some condensation.

Alright, I've finished running around the park at night with a light up frisbee, time for me to take a crack at this.

Beric01, it seemed like a good idea at the time to split the threads between name brainstorming and actual functionality.  Maybe that's a bit overkill, but oh well .

If there's one thing this subcategory system reminds me of, it's the undirected research of Alpha Centauri.  This was interesting, but I don't remember how it felt to play undirected research.  I do remember turning on directed research for most of my games, but I'm a horrible power-gamer .

I've got silly name titles for the subsections, but I basically read down the thread and tried to digest each person's ideas and answer questions as best as I could brainstorm myself.

What's in a Color (Kantok):

Right now, color rarity means a very specific thing: the chance a technology is available when you breakthrough.  Keep in mind that currently red techs hold something like a rarity of between 20 - 30, which means that if you breakthrough 10 times after you've satisfied prereqs,  you are guaranteed to have the tech.  Most of the reds right now do not break the game wide open right now, and honestly I don't think they should.  I do think that they should be valuable and useful, and perhaps characteristic.

Valuable: The technology opens up new inventions which are more potent than similar inventions available in that subcategory.

Useful: The technology is not terribly situational for its category.  For instance, I wouldn't make Beekeeping a red technology with this understanding of color.  An alternative reading of this sentiment: that the technology will be usable for most people who are given the opportunity to discover it.

Characteristic: Defining.  We know the Japanese for their swordsmiths, the English for their ships, the French for their universities etc. - this is what I mean by characteristic.

I think you have Control Issues (Kantok):

The easiest way to take the sting out of the control issues is to make sure the player is most always pleasantly surprised when they breakthrough and get their technology - or at least, that their minions are researching in a competent manner to their wishes. 

I think this will require some under the hood checks, eg. not to give out farming unless the player has control of grains, not to give orchards unless they have fruit, etc.

We Can Rebuild Him, Faster, Stronger, Better than Before (Kantok):

XML is incredibly flexible, and when combined with python scripting you can do some pretty crazy things.  For instance, I have an in-house CIV IV mod right now that adds nand & nor prerequisites, hides technologies you can't currently research, and colors the background of a technology based on its cost.  I'm not a super coder or anything like that, but with some careful thinking, changes like this aren't that big of a deal.

Basically, give us your best ideas, and worry about implementation later!

Note: The Nand & Nor Prerequisites Mod required some .DLL modifications (C++), mostly to add additional XML Tags.

You're Just One Unlucky B*stard, Aren't You (Kantok):

Here is my first truly radical idea: we've bounced around the concept on these forums about gray technologies, techs which you have so far surpassed that you automatically discover them with your breakthroughs.  I've started thinking, perhaps we're thinking of this wrong.  Instead of a tech becoming gray after a period of time, why not have different classes of breakthroughs!  You would pop-up with a message to the player saying: "Congratulations, your scholars have finished their research in [subcategory name] and have achieved [type of breakthrough] rediscovering the following technologies:[][][]", yeah, you read that right, technologie(S).

Dramatic Success: 1 Red Technology and 1 Green/Yellow Technology

Success: 1 Yellow/Green Technology and 1 Green Technology

Moderate Success: 2 Green Technologies

This is pretty huge, and if we went with it, we would definitely need to start breaking some of the technologies apart and deciding what really is green versus yellow, versus red.  I imagine that I would structure the percentage chances in favor of moderate success in a 60:30:10 ratio.  However, this also means we can have things effect your breakthrough types, like sovereign traits and inventors!

Game Over Dude (Iswallie):

The late game is where the system has a hard time coping, though I think a system which uses typed breakthroughs would handle better.  Another way to deal with the late game setup (where you really only want significant technologies, since you've covered the basics which you want already) is to use the gray technology idea which has been bouncing around for a while.

One, Two, Three? Or Two, One, Three? (sjwt):

This works in a similar way to the pulse system I tried to explain in the Technology System Tweak thread, you should read the paper, and not just because I wrote it! .  Out of order technology acquisition is always a little strange: after all, if we've discovered lighter swords, why should my researchers ever waste time with lesser technologies.  I solved this in the pulse system by isolating groups of technologies (Pulses) and only having out of order discovery occur in each pulse.

Now, here's a fun question: How many subcategories is too many?  The degenerate case is to have a subcategory for each technology, but how many subcategories should we be comfortable with.  I think no more than five, to have a parallelism with the technology categories themselves.

Unlocking categories with technology sounds interesting, but then we run into our tree problems again...

Bettick, I didn't mean to ignore you: a wildcard might work, yes, but really, we do want the player to put some thought into which subcategory they want, and I think including a wildcard would be a touch clunky.

Summarizing the General Philosophy:

Research with subcategories is supposed to reflect the player telling his research minions to solve a particular problem, and his minions coming back with a solution.  It may not be the best solution, but it will help solve the problem the player asked them to solve.  The system falls apart if it doesn't, because players become frustrated at their lack of control, or at least, I would.

Therefore, while balancing the tracks, we need to keep in mind that they all need to be valid solutions to the problem the player claims to want solved.

Redefining Color:

Currently, technologies are organized along lines, or tracks (the word used in the XML): linear prerequisite trees.  The player advances along the track, with his researchers choosing tracks based on some reasoning, attempting to solve the problems the player identifies.

I say, that for each step along the line, there should exist different versions of each technology: Green, Yellow, and Red.  Green is the default, Yellow is a little better, and Red is a true marvel.  Which version the player gets should be dependent upon the type of breakthrough they have achieved.

Players following the same tech pattern then, are going to be slightly better and worse at different things.

Defining Your Civilization:

This brings in another opportunity, to tie in the academies and typed research points that were mentioned a while back (a week or so, give or take a few days).  Now, instead of academies giving more research towards a particular category, they would instead increase the chance of better breakthroughs in their associated categories.

I like this idea. Two independent groups of people researching the same technology are probably going to come to different (albeit similar) conclusions. Katanas vs. longswords is a good example that springs to mind.

Definitely like this idea...similar to the creative trait in GC2, but adapted to the system at hand. One could also put side quests in to achieve things that effect breakthrough types. Finding the library in the lost Necropolis of the Ancients yields a higher percentage chance of red techs...but only for as long as you can control it vs the unending undead hordes 

Or your Sovereign could uncover an ancient ritual in which he gives up a few points of Constitution in exchange for some of that ancient knowledge (i.e. higher percentage of red techs)...oh wait I think that storyline was done somewhere

I do not agree with this one. A player should not need direct control over a resource to research it. This is especially true for when the tech is a prereq for others. For example, mining: Say I have control over a quarry which requires Advanced mining to harvest, but I don't have an iron mine. Do I never get to use my quarry then? Do I get to skip Mining and go to Advanced Mining? What if I lose control over the resource I am researching before the research is done?

Also, a player can trade for resources via diplomacy...I may not control any iron mines, I can, however, trade gildars to someone who does...but it does me no good if I don't know how to make things from it (swords and armor) which I couldn't make because I didn't have an iron mine and it wouldn't let me research those techs in the first place. 

You run the very real risk of removing huge swaths of the tech tracks with this method simply because a player has a crummy start position. Immersion factor or not, realistic or not, I wouldn't find it very fun. 

Now, to tie it back into the system mentioned above (Redefining Color)...Having control over a resource gives you a better chance to harvest it more efficiently when you research that tech. For example, if I have access to an ore deposit, and I research mining, I have a better chance (+25%?) to research the yellow or red color mining tech (maybe +10% production for yellow and +25% production for red?). Or perhaps give a bonus on researching the tech in the first place (i.e -25% research points needed). This way a player is not punished for lacking a particular resource, but is (potentially) rewarded for having control over it.

Okay now that I've had my coffee and had time to ponder this...

What's in a color?:  I think I was unclear in what I meant by red techs being society changing.  I don't want them to be drastic, hard to balance things.  I see them more as the epitome of their field.  In the farming category, red techs might be irrigation or crop rotation.  For siege weapons/ballistics it may be the trebuchet or ballista.  Something powerful in such a way that they can change how you handle whatever category they relate to (farming or siege warfare, in my examples). 

I especially like the idea of characteristic red tech's, but that adds balance issues and now you need to come up with a way (not overly hard from a conceptual standpoint, but it's more change) for player created societies to define their characteristics (My people are a seafaring race, for instance) and to have that come into play in little spots throughout the tech tree.  If you don't have it come into play occasionally throughout the tech tree it becomes a superficial choice and basically turns into Civ4's special unit.

Control Issues:  My concern here is that some players just don't like the idea of not having a nicely mapped out tech tree.  The idea of subcategories and getting back an appropriate tech that the game chooses for you further exacerbates this.  By eliminating the "Tech Discover, pick from this list!" that we have now, you're moving even further away from tech trees and the ability to map out where you're going.  Personally, I like the idea of only having Macro control over your research (that's why I proposed the idea) but I know many players won't. 

I don't agree that you would need to be overly careful to make sure that each discovery is directly relevant to the player right at this moment.  That goes counter to the idea that your researchers are looking to develop new technologies or REDISCOVER old ones.  By not guaranteeing that each tech is immediately relevant you also eliminate Icepick's concern that if you don't have a mine you never get mining and hence all techs that are based on mining are now cut off to you until you get a mine. 

You're Just One Unlucky B*stard, Aren't You:  I don't like the idea of different classes of breakthroughs, at least not as presented here.  I think a simpler mechanism would be to "gray out", as you say, certain techs that you've clearly surpassed and that any grayed out tech has a chance to be auto-discovered when you're doing research in its category.  IE. Once you're "Warfare -> Weapons" level is high enough, daggers becomes gray (all of your research has succeed in bow and wooden weapons in this example) and any techs that have "daggers" are their direct prereq become available to be rewarded to you.  If one of those techs are discovered, you're researchers "discovered daggers" in the process of figuring out how that next tech and you're presented with both.  "Sire, we discovered how to make short swords!  In the process we also developed the ability to make really tiny ones called daggers!"  This removes the chance for the game's rng to penalize the players through a run of bad luck. 

This also helps the late game "find specific tech" problem that Iswallie mentioned.  The more advanced of a warfare society you become, the more likely you are to have nailed all the early basic techs (because they were either discovered early or auto-discovered).  If there are dependencies between techs (even across subcategories) this means that as you auto-discover the basics through your advanced research you're also satisfying the dependencies and opening up further research. 

I think the idea that you get to late game and all the sudden want "that one specific tech" defeats the whole point of the system.  How is it okay to want that one specific tech in the late game, but not in the early game?  I think once you get to the end of the tech tree in a given subcategory (not that we see the tree, but the game knows it) you're chance of researching specific techs (as opposed to the generic "Melee Weapons II, Melee Weapons III that will be at the end if you want to keep incrementally improving your ability in a given area) should increase so that within a few research cycles you've researched all the available techs and all that is left is the incremental improvement techs. 

This graying out also eliminates the 1,2,3 or 2,1,3 technology problem because once 1 is grayed out and you discover 2, you're given 1 as a "hey we worked this (1) out in the process of discovering (2)".  That leaves it up to the player to decide if they have a use for tech (1) or not and if not, they just let it sit and go about using (2).

I'll post more response in another post in a bit.  After I have another cup of coffee. 

Edits:  Should have had the second cup of coffee first.

A change to the typed breakthroughs and then a diagram.

Success - 1 Green Technology, 1 Gray

Moderate Success - 1 Green, 1 Yellow (Orange in Diagram), 1 Gray

Dramatic Success - 1 Red, 1 Green, 1 Gray

The idea is to always keep moving forward in the lines.

Things which are grouped in a column all have the previous column as a prerequisite.  Eg. Hybrid Crops & Advanced Irrigation both require farming only.

Characteristic:

With typed breakthroughs, you could push higher chances of better breakthroughs in particular subcategories.  You'd want to shift the odds in such a manner that the player notices the change.  Eg. my faction trait is seafaring, so when I'm researching in the seafaring subcategory of civilization, I am more likely to have moderate and dramatic successes.

Another way to do typed breakthroughs are to only have one technology at a time, and then we're just adding names to the technology color categories.  I think the X + Green system is better for the stronger breakthroughs because it emphasizes that you are making both progress and getting extra benefits, either from luck, characteristics, or because of other factors which the player has control over (like the academies idea).

We can also fold in a gray technology system here by adding 1 gray tech to each type of breakthrough.  The end result is that your researchers come back to you, and instead of saying: "We discovered X", they can say "We looking into [Category] and figured out how to do X,Y,Z".

Control Issues:

Let's set aside people's uneasiness with having only macro control of their research for a brief moment, and come back to it later.

RNG Management:

I think the biggest problem with the gray system is that people haven't really acted that keen on an endgame technology setup which converges for each category - such systems work, but they're a little bland (CIV).  Should it converge towards similar results (eg. two warfare specialized factions will have access to most all of the weapons and be specialized in one or two each)?  I think if we keep the red technology flavorful and out of the prerequisite line, we'll have enough differentiation that it won't be a problem.