7 people like this
#johncarmack #archive #.plan #jumbotweet
John Carmack's .plan Archive in one jumbotweet!
(Covers 1996 -> 2010)
This was grabbed from http://floodyberry.com/carmack/plan.html, so thanks for making it available!
-----------------------------------------
John Carmack's .plan for Feb 18, 1996
-----------------------------------------
* page flip crap
* stretch console
* faster swimming speed
* damage direction protocol
* armor color flash
* gib death
* grenade tweaking
* brightened alias models
* nail gun lag
* dedicated server quit at game end
+ scoreboard
+ optional full size
+ view centering key
+ vid mode 15 crap
+ change ammo box on sbar
+ allow "restart" after a program error
+ respawn blood trail?
+ -1 ammo value on rockets
+ light up characters
vsync on high framerates
-----------------------------------------
John Carmack's .plan for Feb 19, 1996
-----------------------------------------
* fixed vanished bmodels at startup
* fixed getting stuck problem
* fixed the sound out of range problem
* hack patched the dissapearing under water
+ remove look spring
+ fix radius damage
+ too many people on status bar
+ invis in water
+ multiple pain sounds started
+ shift jumps while in console
+ less blood time
+ gone armor icon
+ look up/down keyboard modifier
clamp velocity on respawn
clear damage flash on respawn
bad changeup
bob speed kept on restart
no puffs in sky volumes
id copyright seal
teleport screen warp
-----------------------------------------
John Carmack's .plan for Feb 20, 1996
-----------------------------------------
* rewrote sound packet communication
* fixed multiple pains from a single shotgun
* qcc -copy option for sounds and models
* fixed up pak file support
* removed .rc files
* screen size over status bar
+ monsters need to be pushed into teleporters
+ blue frag pieces
+ NaN impact time is a crasher
+ looking down on weapon bounces away from bottom
+ timing preventing attacks at startup
fix hardwired vid_uopdate
remove redundant death sounds
teleport warp
torches
cpuid?
monsterblock entity
multiple sky textures in a level
-----------------------------------------
John Carmack's .plan for Feb 21, 1996
-----------------------------------------
* frag bar
* start profile on late frame
* fixed relinking bug
* improved prediction bounding
* change weapon chooses supers first
* ammo pictures on status bar
* drop pain flash again and bound
* tweak armor and spike damages
+ don't pick up equal armor
+ ceiling doors don't cause damage
+ demons crash when jumping
+ monsters need to use bsp clipping
+ need to save memory...
+ Fatal error: NaN impact time from entity
+ change armor damage
+ drop weapons
+ face to face with nailgun does no damage
+ +pointfile from cmdline doesn't work
+ conglomerate hunk printing
+ shoot triggers
+ missing a Z update on rockets sometimes?
+ firing after net games
+ negative scores on frag bar and frag board
+ world backpack and armor model
+ weapon bounce down
+ center look key
+ check change levels in high res mode
show ammo counts in single player frag screen
radius damage should open secret doors
five point font on kills
rocket poke into walls
shoot open connected doors
gib bounce sound
-----------------------------------------
John Carmack's .plan for Feb 22, 1996
-----------------------------------------
* fix incorrectly updated origin bug
* added alias backpack and armor
* fixed weapons sticking into wall
* changed "connected to server" to "connected to map"
* fixed console
* armor skin numbers
* fixed "not shooting" bug
* changed sound back to origin based instead of entity based
+ swimming directions
+ quit scores are wrong
+ screen flicker crap
+ radius damage sight check
+ crushing issues
+ bodies stopping in midair
dimmed out weapons
follow killer
spin head inside gib
-----------------------------------------
John Carmack's .plan for Feb 23, 1996
-----------------------------------------
* fixed sound origins again
* fixed crushing stuff
* added "wait" to triggers
* added startup stuff for demo
* swimming direction stuff
* added '+' sizeup as well as =
* changed "unnamed" to "player"
* fixed skinnum clearing
+ gibbs need to fall after trap doored
+ ambient
+ make bubble sound while swimming forward
+ server speed
+ case insensitive on key names
+ hunk alloc
+ nans
+ better notify scrolling
+ final score dumping
+ negative frags
+ reconnect problems
backpack dropping underlava
"hostname" makes gane think it is a server
swimming models
weapon cycling
dedicated server stuffcmds
-----------------------------------------
John Carmack's .plan for Feb 24, 1996
-----------------------------------------
* added case insensitive key names
* skipped license check on dedicated servers
* changed prog score sorting some
* took out autofire
* fixed negative frags
* cleaned up connection management
* fixed hunk_alloc
* no sutch alias frame problem when leaving server
* "map test1" on system with enough memory
* flickering side crap when level restarting
* added base directory to path
* changed heads to tosstop
****** RELEASE QTEST1 ******
+ unupdated Z problem is still in there
+ flickering weapon icon
+ blood trail to respawn
-----------------------------------------
John Carmack's .plan for Feb 25, 1996
-----------------------------------------
* emit bsp files
+ idle loop on disconnect
+ key sticking
+ fix noclip in the water
+ modelgen scale option
+ run a single level for a long time, then connect hangs
+ grenade radius check
gibs stuck in walls in low G
flash blinding
turbulent texture direction
space in smiley talk message
talk sounds
full chat mode
-----------------------------------------
John Carmack's .plan for Feb 26, 1996
-----------------------------------------
* ogre sounds
* made 180 turns go left for better ogre dragging
* error checked and extended qbsp
+ drop green slime percentage
+ ram chip overlapps turtle
+ do DEC quote
+ nolerp update
caps-lock = full speed
no particle effects on sky
-----------------------------------------
John Carmack's .plan for Feb 27, 1996
-----------------------------------------
* expand qbsp to 32 bit values for edges and faces. sigh.
* fixed key sticking problem
seperate mouse sscaling values to allow negative up/down reversal
frag bar shows first two digits, not last two when >= 100
randomize head direction
slime less damage
drop to knees when crunched
check deathmatch flags before precaching
-----------------------------------------
John Carmack's .plan for Feb 28, 1996
-----------------------------------------
+ upper health bound
+ keyboard freelook
+ more air acceleration
can a hostile monster hit it's movetarget and turn away? check scirk and smotte don't cache player model in single player "fair" mode with syncronous clients kill timers on all spawned objects
-----------------------------------------
John Carmack's .plan for Feb 29, 1996
-----------------------------------------
* increased air acceleration
+ don't allow up/down move at all without noclip
+ weird fall out of level in sbad
-----------------------------------------
John Carmack's .plan for Mar 01, 1996
-----------------------------------------
* added ping command
* fixed time warp from overloaded dedicated servers
* rotate binaries
* respawning items
+ clean model flags on remove
+ tim's color
+ weapon ammo not redrawn sometimes
+ no sutch alias frame after gibbing
+ give more feedback during the connection process
+ set absolute screen size command
safe allocation failures from vid / zbuffer
send full pathnames from server to client, not findfile
make all commands remote if priveledged
unique names
inquire about server rules (teamplay, fraglimit, timelimit,
maxspeed, gravity, ticrate)
is vid still flushing the entire screen?
empty path causes error in quake?
don't need to update frame of target entity
-----------------------------------------
John Carmack's .plan for Mar 02, 1996
-----------------------------------------
* wrote sgi test demo
-----------------------------------------
John Carmack's .plan for Mar 03, 1996
-----------------------------------------
* added precache_file qcc option for -copy files
* capped health at 250
+ solid, invisible brushes
+ don't send linear entities in signon message
+ write a message trace printing mode
+ is 256 angle quantization from client hurting us?
+ sky animations use the wrong time base
+ centralize the connection process
weapon switching is wrong
no twitch in god mode
if a linear update entity doesn't get an update in a packet, remove it
bound upper client message sends properly
RELEASE: bump protocol version numbers just before releasing
-----------------------------------------
John Carmack's .plan for Mar 04, 1996
-----------------------------------------
* got new internet code running
* server protocol number
* stopped printing name change message for first message
* put usleep back in
+ an entity model is being cleared out from underneath it
+ rotating body parts
+ binary bit fields
+ damage in datagrams
+ push back at edges when moving slow?
+ twirling gibs
+ bit fields for binary objects
+ never reuse binary object slots
+ fix connect to a bad map
+ noticed water to dm only clipping prob in tim13
+ no sutch alias frame problem
magic8ball.rogue-ent.com = 38.252.190.8
ability to make custom plat activation triggers
-----------------------------------------
John Carmack's .plan for Mar 05, 1996
-----------------------------------------
* basic zombie actions
* added dog
* fixed bug in QuakeEd that caused a crash when setting a bad classname
* changed health stat to a short to fix negative numbers
* added binary object bit update
+ zonbies need to change size when down
+ secret doors need to crush damage, and return to open
negative health
fish have problems
cycle general updates in wasted badnwidth
-----------------------------------------
John Carmack's .plan for Mar 06, 1996
-----------------------------------------
* fixed all network play issues
* better monster combat
* monsters don't activate plats
* shambler shoot up
* stubbed enforcer
* soldier standing frames
* monster use function makes them hostile towards activator
+ drop ammo
+ monsters see farther
+ negative ammo
+ extra friction near edges
+ fracs aren't right in single player
+ use monsters = wake up
set ideal yaw upon first sighting
temp object updates
check size on teleport destinations
player only teleporters
puff of smoke when shooters fire
weird high look up on romero's level
step on grenades
flyers look down
eagle eye checkbox
monsters not trigger plat
flying change height
monster jumping
make tents for shotgun puffs
leave level running too long problem
shambler range
-----------------------------------------
John Carmack's .plan for Mar 07, 1996
-----------------------------------------
* increased max portals on edge in bsper
* NT gl testing
* fixed path walking
+ clear static objects for restarts
-----------------------------------------
John Carmack's .plan for Mar 08, 1996
-----------------------------------------
* dropping ammo
* zombie gib non-solid
* all zombie attacks
* scrolling notify text
* shooters that fire superspikes
* keys
* increased monster sight distance
* fixed noclip in water
+ fix sound code
+ item only in dm flags
+ shoot through complex bmodels
+ NaN stuff
+ change impulse over to impule # instead of impulse1
+ dog frames
+ demos!
fix noclip underwater
all zombie pain issues
duplicate impulse events
status bar art needs work
jrbase3 teleporter can be hit behind dmonly
do pain well on all monsters
don't blit the entire screen when sized down!
clean up headers
-----------------------------------------
John Carmack's .plan for Mar 09, 1996
-----------------------------------------
* wrote cl_shownet 2 dump
* rebuilt qcc to take $scale parm
* worked on demon
* cleaned up obsolete utilites
* fixed super shotgun size for sbad
* moved wadfile to hunk
* added colormap grab to qlumpy and got rid of vid_256.h
* finaly gave dec quote
* spread monster thinking times
+ fix angle on spike shooters
+ pick up super nailgun == illegible server message?
+ damaging trains
+ sandy: sally, don't put all the weapons on it, and help the PVS at start
+ get rid of surface->drawnframe now?
fix weapon switching
fix sound mixing
climb out of water
weapon pickup can overmax ammo
staticly define sound arrays
lightning client entity
fly move to goal
checkbox
stop repeat jump
check DAMAGE_YES on dead bodies
func_oscilating
teleport warp
is surface->texmultab[MIPLEVELS] worth the space?
combine surface->texture and ofs into texinfo?
-----------------------------------------
John Carmack's .plan for Mar 10, 1996
-----------------------------------------
* fixed static object update bomb
* fixed crazy speed guys
* allow restart after objerror
* fixed bug with restarting droped out of world messages
* spread standing frames more for groups
+ remove precache_item
+ ogres that won't wake up
+ spawn flags
pause turn speed
army guys don't wake up by touch?
jump velocity shouldn't make head bob
-----------------------------------------
John Carmack's .plan for Mar 11, 1996
-----------------------------------------
* fixed backpack dropping in deathmatch
* fixed monster sighting PVS problem
+ fix dos qcc
+ demon landing frame
+ trigger activated train
+ enforcer
+ dog
+ -1 ammo, 1 ammo on super shotgun, etc
+ invisible brushes for clip hull
+ grenade launcher shouldn't aim move
update ammo when picked up grenade launcher with rocket running
damage value on trains and secret doors
player stayed on console after leaving
jctest1 door fuckups
grenade open secret doors
-----------------------------------------
John Carmack's .plan for Mar 12, 1996
-----------------------------------------
* got new alias stuff sort of working
+ pause message needs to stay up
+ change level trigger
+ knight bodies are solid
lighting hidden objects
level loading pacifier
weapon switch button
doors that stay shut on some multi levels?
mouse button stuck down
monster shouldn't get key door messages
-----------------------------------------
John Carmack's .plan for Mar 13, 1996
-----------------------------------------
* got new alias stuff (mostly) debugged
* optimized BoxOnPlaneSide
* stoped generating backwards planes
* demon jump higher, fixed melee attacks
* made knight corpse non-solid
* crunched progs.dat file structure
* culled unused edict fields
+ shoot trigger
+ alias clipping still isn't right
+ reverse dmonly / not_dm? combine with spawn only flags?
+ work on sound code!
+ fixed secret doors stop working
+ make objects fall when floor is removed
make #ifdef PARANOID checks
lightweight entities to save space?
grenade / rocket damage to triggers
-----------------------------------------
John Carmack's .plan for Mar 14, 1996
-----------------------------------------
* new trigger code with delay
+ jrbase3 spawn problem
jump triggers
dragon path
uncache player on single player
monster missile checking for race relations
gib pack
wizard partical trail
multiple sky maps
non-bleeding takedamage
-----------------------------------------
John Carmack's .plan for Mar 15, 1996
-----------------------------------------
* made an execution stack for progs
* spawn flags
+ fix NaNs!
+ proper level changing
door no backup option
PVS for net update
check entity relinking for excess
good tempents.
-----------------------------------------
John Carmack's .plan for Mar 16, 1996
-----------------------------------------
* made -1 wait doors not return when blocked
* made secret doors and trains do damage
* added edge friction
* fixed reset self bug that caused jrbase3 not to load
* loading pacifiers
* split out some of the physics code
+ do from-the-beginning demos
+ why are NeXT fullbright colors still bad?
+ not starting up at american's house?
+ move the if(deathmatch) remove (self) statements before monster
precaches
+ trigger start trains
shambler and wizard need idle frames
bump all version numbers before final release, so no maps work with
qtests!
general sound objects?
mouse button sticking
win95 startup with map
eliminate MAX_FACES from qbsp? (johncm1.map)
-----------------------------------------
John Carmack's .plan for Mar 17, 1996
-----------------------------------------
* triggered trains
* began work on sound code
+ edge friction jumping
+ edge friction in air?
ambush guards
temp ent precache
does pacifier not work on net games?
-----------------------------------------
John Carmack's .plan for Mar 18, 1996
-----------------------------------------
* fuxed button triggered delays
* added classname to light error message
* demon more damage
* sounder objects
* light_switchable
+ grenade proximity explosions
+ fix jumping off edges
+ wizard fast shots
+ target lights for controling them
+ plat hit on head bug
+ ambient light
spiral the view when gibbed
level based gravity
-----------------------------------------
John Carmack's .plan for Mar 19, 1996
-----------------------------------------
* switchable lights
+ changed light program to unique switchable styles
+ target usable monsters
+ toggleable doors
+ leaf lighting
+ F2 reload crashes
+ grenade radius
+ spikeshooter directions
+ sound cache shouldn't be cleared all the times
+ demon jumping
+ demon death frames
+ chain activators for monster deaths
+ crawl out of water
+ ambient from map
autosavemap?
different teleport flash
linked doors still have problems
work on swimming
control triggers for player only, or with monsters
better knight code
turn off regular shooters when used?
once trigger on door_start_up?
ctrl-alt-delete reset?
remove r_fullbright in deathmatch
sound streaming
fix "couldn't allocate Z buffer" error
reduce sound latency
-----------------------------------------
John Carmack's .plan for Mar 20, 1996
-----------------------------------------
* deathmatch onyl spawn doesn't work
* drowning damage
* changed QuakeEd to get progs from id1_
* added colors to QuakeEd entities
* got rid of /lquake, moved QuakeEd caches to /qcache
+ don't take datagrams until acked second message
+ not-dm is intangible
+ sally torches not spawning
+ client called twice?
+ combine damage messages from super shotguns
+ gib zombies lying down?
+ more water friction
+ swimming bubble noise
+ light the weapons differently in bright areas
+ keyboard freelook
+ move QuakeEd start position to playerstart
stop wall climbing
sleeping pause
raise rocket launcher with 0 ammo
-----------------------------------------
John Carmack's .plan for Mar 21, 1996
-----------------------------------------
* fixed quakeed bug to allow carrying selection to new map
* fixed updated static bug
+ demons that fall through world
+ "wait" exec command
+ end of line exec bug
do style lighting for alias models
for release: Id Trademark graphic
-----------------------------------------
John Carmack's .plan for Mar 22, 1996
-----------------------------------------
* fixed bug with "alias" command that failed to add a newline
* fixed bug in command execution that sometimes mashed commands together
* added "impulse #" command to replace impulse0 impulse1, etc
* allowed "bind" and "alias" to grab the rest of the line, not just arg(3)
* don't clear impulse events every frame, so a weapon change will wait
for attack completion
* added "wait" command. These changes allow:
bind g "impulse 5 ; +attack ; wait ; -attack ; impulse 2"
* got demos mostly working
+ worry about shutting down listen servers before connect / demo playback
+ remove item precache
+ finish switching lights
+ keyboard freelook
+ do clients need keepalives?
spikes coming through walls
explosive nails
change thresholding on trimapper in higher res!!!!!
muzzle flashes
-----------------------------------------
John Carmack's .plan for Mar 23, 1996
-----------------------------------------
* lots of cleanup in cl_ code
* killed looping sounds at disconnect
* faded palette back down at disconnect
* fixed bobj miscount and added bobj count error check
* interpolated angles and velocities for low update connections
* finished demo code
+ sound caching....
+ looping sound still doesn't work on my system
+ can't throw out spawned sounds too far away, because of teleporters
+ ammo box on sbar doesn't allways update after death
+ two state door
+ allocate translations on the hunk to save single player memory
remove headless / legless death frames
swimming frames
server viewpos problem
relinking by size still isn't right
explosion core
precache_auxsound for non-essential
don't cache player model in single player
-----------------------------------------
John Carmack's .plan for Mar 24, 1996
-----------------------------------------
* level to level transitions
* finished light modifications for switchables
* fixed bug that caused some entities to not appear
+ weird field trashing bug
+ don't clear sound precache on non-progrestart
+ stuff not showing up in jccode
+ savegames
+ nail angles are wrong
+ cought on buttons in johnc99
+ is anything confused by sv.active on clients during a net game?
stereo positioning is wrong
nograv
handle coop games
make lightning a tent
worry about security risks with pathnames for demos and savegames
need to save lightstyles in savegames
non-convex bsp output
-----------------------------------------
John Carmack's .plan for Mar 27, 1996
-----------------------------------------
* fixed soundlist
* explosion through walls doesn't damage
* monsters don't melee through walls
* fixed memory trash from restarts
* changed signon messages to more client requests, removed buffers
* invisible clip solids
+ wake up monsters through walls
+ is vid palette setting fixed?
+ remove up/down movement from non-noclip
+ entity bsp clipping for shots and monster moves
+ make QuakeEd center on start position
+ climb out of water
+ light switches?
bsp sticking point
no aim key
don't include trigger and clip mip textures in files?
-----------------------------------------
John Carmack's .plan for Mar 28, 1996
-----------------------------------------
* fixed efrag allocation on restart (really)
* fixed static sprite placement
* fixed light casting from torches
* fixed spawning angles
* fixed savegame and multi player baseline object startup
* made QuakeEd go to info_playerstart at level load
+ fix damage trace through entities
+ keyboard freelook
spinning gibs on floor
very high framerates mess up
extra back attack damage
buffered IO problem on romero's machine
stereo still isn't right
partical puffs
-----------------------------------------
John Carmack's .plan for Mar 30, 1996
-----------------------------------------
+ dynamic lights....
+ grenade activation size
+ entnum optimization
+ make progs use pointers for edicts?
+ sideways spikes
esc = floating menus, with console option
remove avelocity?
track down the _SCR_UpdateScreen <cycle 2> [23] from profile
clean up search path syntax
fix style lighting on alias models
more #ifdef PARANOID
-----------------------------------------
John Carmack's .plan for Mar 31, 1996
-----------------------------------------
* changed edicts from indexes to pointers
* fixed byte order problem in qcc
* fixed backward movement dies after many savegames
* fixed bus error in edict linking after a model remove
* cleared rotate flag on CL_SetModelFlags
* made items move with pushers
+ animating textures
+ vis problems out of water in jrbase2
+ get trail characteristics from model name? (now a model flag)
+ switched lights don't work right after restart
+ aiming at fast moving entities
+ center view key
light filter in quakeed is broken
per-entity ambient light value
don't play door trigger sounds if full up
shoot over fallen zombies
no changeup to grenades
add targets to doors
auto pause when console
player gib packages
null bobjmodel in singleplayer
wizard bombs through walls
drowning time?
shambler lightning
hang when starting a mistyped level (time based?)
old deathmatch rules?
lockup overnight
don't hook keyboard on dos dedicated server
cycle edict allocation so spikes aren't reused frequently
make spikes rotate
-----------------------------------------
John Carmack's .plan for Apr 01, 1996
-----------------------------------------
* added $flags to models for client side stuff (rotations, trails)
* allowed coop play
* started using Host_Error for client errors
* lint
+ animating textures
+ NULL connect
+ connect while in a game crashes
different sfx cache size for client only?
frame chnaging while paused?
item targets
respawn in coop
-----------------------------------------
John Carmack's .plan for Apr 02, 1996
-----------------------------------------
* exploded wizard bombs on death
* server optimization
* fixed angle on loadgame and coop
* average monster velocity for aiming
* made wizard fall properly
+ running into nails
+ can still get shot out of the world
+ dynamically allocate color translation tables
+ pause work
+ backpack dropping
water stair climbing
check door linking , regarding the partial closing
restart level spawn parms
dead player save que
quakeed magnified texture view
ambient + gravity attributes
load fames
restart without killing server
no-aim option
-----------------------------------------
John Carmack's .plan for Apr 03, 1996
-----------------------------------------
* made func_wall and removed the dm_only / not_dm
+ sky fuckups
+ single pixel movement quantization?
+ falling out of world
auto aiming sucks
remove superscaling
s->relasecount error
.cfg default for exec
frame too fast bug causing lockups
last request/reply
registration trademark!!!
explosion cores
check side to side movement for big attacks like the shamblers?
change backpack content messages
combine model and sound cache!
better PVS with more leafs?
-----------------------------------------
John Carmack's .plan for Apr 04, 1996
-----------------------------------------
+ teleport flash is broken
+ order of monster fall to ground
multiple input drivers (hard coded)
save cfg
bounce head gib?
wall scaling is step up?
-----------------------------------------
John Carmack's .plan for Apr 05, 1996
-----------------------------------------
+ sound loop points
multi life / level demos
precache_sound return string value
make jump buffered edge triggered
clipping hull lips
hang after sitting overnight on dos
-----------------------------------------
John Carmack's .plan for Apr 06, 1996
-----------------------------------------
use PVS to speed line of sight checks? Is that any faster than doing
the line check, because it involves two point checks?
swooping ai movement by checking distance to impact with forward trace?
make minimum frame time a cvar, so you could clamp to 10fps
runmove() builtin for progs?
draw triggers debugging option (z buffer it)?
look towards killer when dead
-----------------------------------------
John Carmack's .plan for Apr 07, 1996
-----------------------------------------
* fixed tfog
* fixed removal of entities over net
-----------------------------------------
John Carmack's .plan for Apr 08, 1996
-----------------------------------------
* fixed stray rocket trails
* fixed coordinate integralization problem
* identified the Aim() time sink
* fixed smooth step up
* got new clipping code running
+ sound looping prob is due to decimation
+ expand numtextures to a word from a byte
+ remove FL_ONGROUND after being pushed
+ still damage through walls
+ counting trigger
+ paused graphic
+ demon freeze jump
add s and t vectors to the plane description in .map files
remove the pad around bmodels bounds
make pushmove area check instead of linear search
server pass full pathnames to client
dos crashes overnight (timing wraparound bug?)
quake logo screen needs filtering
shoot all the way through walls with objects
wizard fireball
plat problem in tim17
shambler lightning
weird grenade launch spot
auto pause on console
hold "other" on delay triggers
gib stuff
-----------------------------------------
John Carmack's .plan for Apr 09, 1996
-----------------------------------------
* fixed crash when -NXHost on quake
* added +precache 0 option to speed up WAN loading
* fixed sound loading bug
* added loop lengths to sound loading
* increased MAX_MAP_ENTSTING
+ quakeEd filter brushes
+ 16 bit looping sounds hang the dos game
QuakeEd showed "Helvetica-Bold" in entity comment field. Scary.
maake treading water better
trigger to turn off spikeshooters
save the entity string memory
blocked bmodels are conveyors?
-----------------------------------------
John Carmack's .plan for Apr 10, 1996
-----------------------------------------
* made no-light levels auto fullbright
* finished second clipping hull
+ zombies on the ground can't be gibbed
+ listen servers not working?
error check all strcpy in QuakeEd
slow slide clipping with ogre hull?
grenades should trigger shootable triggers / objects
looping sound that didn't stop
-----------------------------------------
John Carmack's .plan for Apr 11, 1996
-----------------------------------------
* fixed damage through walls
* fixed gibbing downed zombies
+ KEY_SEMICOLON
pointing?
ten second max looping sound?
integral sound stepping?
the damage byte stream is what is causing packet overflows
debounce jump
is there a mouse call to find the number of buttons?
fix palette after vid change...
too many vid_palette calls
DON'T FLUSH THE ENTIRE DAMN SCREEN!
fix snd_next to be snd_none
change sys_loadfile to com_loadfile
"system is little endien" message as optional verbose debug
sometimes shotgun sound doesn't play
double doors independantly closing
server speed went back up
fix auto aiming
no gun aiming
no change to grenade launcher
ogre grenades
zombie gibs
r_ambient
fixed angle for grenades when stopped
client side shambler lightning
gravity on corpses pushed into blocking positions
slime darkness value
sky time problem
integrate dirpack with qcc
check precedence of test directories and pak files
rocket auto aiming is not working right
amtele2 teleport problem
no save when dead
memory fail to console, not dos
still overrun nails
-----------------------------------------
John Carmack's .plan for Apr 12, 1996
-----------------------------------------
<at raven>
-----------------------------------------
John Carmack's .plan for Apr 13, 1996
-----------------------------------------
<at raven>
-----------------------------------------
John Carmack's .plan for Apr 14, 1996
-----------------------------------------
* fix 8 bit palette in NS
* modified modelgen to allow multiple models in one file
* fixed qbsp to detect cases of no entities placed and skip fill outside
* new wizard spike attacks
fix corrected palette at mode change
make dos utility tree
make QuakeEd's menu commands variable length
debounce console
debounce jump
remote slist
slist with user names
fall out of world when dead
s_releasesound: s->refcount < 0
sound cache parameters
clear pause on restart
check pak file overriding
-----------------------------------------
John Carmack's .plan for Apr 15, 1996
-----------------------------------------
* lowered slime cshift
* wizard slide move
* dog frames
* fixed auto aiming when dead on
* soldier less hp + tuning
* various gibs
* dynamically allocated scores structure
* trigger_counter
* door TOGGLE
* fixed forking bug that trashed pointfiles
* fixed sky fuckup
+ fix demon pop jump
+ zombie get up inside other
+ zombie gib throwing
+ lose onfloor when pushed
+ animating textures
+ schurch weirdness
make light styles global prog vars
base4 entrance door sucks
back attack damage
fix dynamic light values of players
aim up at nearby monsters
load pcxs in lbmlib
text triggers
swim slow straight up?
temp entities
client sound calls
zombie heads bouncing
bouncing monsters out of the world
gib bodies between pushers
-----------------------------------------
John Carmack's .plan for Apr 16, 1996
-----------------------------------------
* reduced bodies to points on squash
* turned off onground when pushed
* new demon jumping
* fixed zombie stand ups
* fixed zombie standing off floor
* fixed uninitialized vector in movestep
* fixed scr_ofs cheat
* fixed upjump cheat
* _field names are ignored by quake
* fixed bottom checking
* fixed up fov
* moved icons into view screen
* added pause icon and showpause cvar
* renamed SOLID_EDGE to SOLID_TRIGGER
* delayed monster drop to floor
+ mbug.map texture toastage
+ save / loadgame start view angle
+ fix demon popjump
+ flickering ammo box on restart
+ weapons not following to next level?
+ knights not coming off paths when angered
+ load / save games are broken
+ pause veloicty spin
+ reset lightstyles on level load
more screen turbulent warps
default fields in /*QuakeEd header
is draworder working in deathmatch?
enforcer no melee attack, side step
triangle transition size
open secret doors with area effects
alias model racing stripe
shoot over grounded zombies
activatable trigger
wizard altitude changes
enforcers need to drop ammo
player torso gib shouldn't be used for most monsters
teleportation issues
some keyboard keys make it in while changing levels
make a base directory variable for all util scripts?
invisibility (remove weapon)
flyjumping up slopes
better polygon lighting
track killer when dead
allocate cl_entities on the hunk
start dead zombies
cvar changing buttons
invulnerability powerup
raise / lower weapons
is qc pausetime needed?
delay item drop to floor?
run first second of game before entering player
-----------------------------------------
John Carmack's .plan for Apr 18, 1996
-----------------------------------------
* finished new bsp portalization process: 20% faster, 1/5 the memory
* fixed monster repeat teleport
* klook and mlook
* fixed monster stepping up into mid air
* what breaks switchable lights is a bsp->entities without running light
* changed network startup to use signon ready flags
* added extra face-on wall friction
+ fix ammo between levels
+ dark knight
+ message wasn't a nop
+ schurch zombie graves
+ sandy's bug1
+ slow drifting spikes
+ kill wizard spikes after their death
+ s_refcount bug still exists
+ wizard spike starters spawn inside walls
+ plat dmg fields
+ longer pause for ogre first shot
should demons step up before leaping?
shooting spikes out of rockets
need mouse configuration in the menus
try removing bounding pad on bmodels
add fadable light switches
ability to bind specifically to key up / down ?
bound ammo on weapon pickup
lava damage for monsters
error check add to reliable messages
less accurate ogre tossing
-----------------------------------------
John Carmack's .plan for Apr 19, 1996
-----------------------------------------
* fixed world/entity clear allsolid flag bug
* tracked down problem with zombies popping out of graves
* fixed path follow when in combat mode bug
* fixed standing on top of monster stuck bug
* animating textures
+ don't report surface cache size when caching sounds
+ pick up items multiple times when on dropping plat
+ pause continues across restarts, but plaque doesn't
+ overnight lockups
+ debounce jump
+ wizard spikes start in walls
overbright light palettes?
slope auto pitch needs work
secret area counters
don't slide on slopes
zombies need to be gibed by pushers
monsters trigger plats
trigger_notouch
warnings on coplanar brushes in bsp?
only disable auto center when head is under water, not feet
debounce escape
lightstyles in savegames
sticky bsp corners
wizard partical centers
make triggers use BSP models?
better sky management
-----------------------------------------
John Carmack's .plan for Apr 20, 1996
-----------------------------------------
* added hell knight
* Hunk_ReportSurfaceCache() instead of automatic
* fixed multiple touch on items riding plats
* helped non-solid and trigger items push better
* cleared pause on map restart
* fixed level change parms
* dynamic lights!
* special wide box for missiles against monsters
+ larger rocket radius
+ slow motion screwups
+ nailgun stops rotating when riding a plat
+ stuck roll angle
use knight kneeling frames
zombie crucified frames
looping sound while invisible
make running direction changes smoother
error check all file opens?
load temp file on stack if possible?
remove SOLID_SLIDEBOX
put auto aiming back in?
move rule checking to end of frame
quicksave / quickload
dissallow spawn parms on deathmatch
make connect to server send version number
make temp entities auto remove
no body que?
water up swim speed
tim was getting stuck in deathmatch
-----------------------------------------
John Carmack's .plan for Apr 21, 1996
-----------------------------------------
* fixed texture animation cycling roundoff bug
* fixed up missile clipping
* fixed super spike -1 ammo and use 3 ammo bugs
* checked deathmatch before precache on monsters
* jump out of water
* debounced jump
+ check extranious messages on connect
+ continuous looping sounds
+ button texture changing
+ state textures for buttons?
+ remove the legless / headless player death frames
why is shambler lightning funny colors under NS?
extra point radius shoots through walls
big sound cache for deathmatch
abort load command?
is max_health ever used in progs?
don't go to frag scoreboard when single player dies
clipping against surfaces causes multiple slime damages in multiplayer
only?
treading water less bouncy
jetpack model is saved wrong
super size boss
-----------------------------------------
John Carmack's .plan for Apr 23, 1996
-----------------------------------------
* changed network update to client PVS specific
* fixed broadcast svc_setangle on player connect
* fixed "Message was extranious" connection bug
* partical fields
* invisibility model switching
* effects passing from server
* invincibility
+ self.modified is no longer needed with PVS updates
+ empty space back fill Z buffer bug
vertical aiming sometimes shoots at a farther object
over long latency lines, should shots come from the position the player
was
viewing, or the true position?
combine client->active and client->spawned into a client->state
backpack message should include weapon gainsed
throw monsters around from explosions
drop wizard by half gravity
make tri subdividing rasterizer based on screen size! (huge models go
slow)
american got a prog to crash
allow listen servers to have a sys_ticrate?
cut back on use of global host_client
new teleport surface warp
body que is broken
-----------------------------------------
John Carmack's .plan for Apr 24, 1996
-----------------------------------------
* muzzle flashes
* eat keys between levels
* added new pain frames for soldiers
* fix savegames with PVS update
* new status bar updates
* fixed packet overflow and noclip with PVS update
* synced zombie throws from exact position
* put in new ogre grenade
* no span lighting in subdivided triangle code
* better gibbing code
* vid_size for nextstep
* determined the hang is in the sound mixing loop never exiting
+ auto pause with console
+ more time for ogres before launching grenades
+ is restart doing a progrestart?
+ fix gib head bbox
+ head outside level
+ backpacks don't drop when floor is pulled out
+ movetype_missiles need to be intangible against owners
+ gib dogs
+ teleport flashes aren't in the pvs
+ soldier3 new pain frames
+ fix high framerate bug
minimum ambient light field for bmodels
make it easier to jump out of water
default water treading
better discrimination of slime and water
need to cycle allocations of temporary entities to prevent false reuse
repeating bounce sound on dropping plats
crush monsters extremely by plats
about to drown warning noise
load last savegame by default
monsters burn in lava
float particals from tport pads
don't switch to frag bar on single player death
slope lookup
got stuck in a downed zombie
make an error message in qcc for function calls inside a parm list
polygon torches
need to make impulses reliable, or looping sounds may not be killed
restart game with entrance spawn parms
allways test demos and savegames after a net change!
option to match ping times for fair deathmatches?
get bonus items on top of doors spawning deterministically
selective cheat for testing
-----------------------------------------
John Carmack's .plan for Apr 25, 1996
-----------------------------------------
* increased spread on super shotgun
* rewrote sound loading and mixing
* fixed sound refcounting bug
* fixed sound not playing bug
* fixed sound hanging bug
+ axe
colored lighting sample
lightning gun
light bmodels
don't let the view model go full dark
-----------------------------------------
John Carmack's .plan for Apr 26, 1996
-----------------------------------------
* bringing down console auto pauses in single player
* fixed ultra-slow motion console movement bug on bad bat file startup
* fixed slow motion when > 72 hz framerate
* added axe
* never change up to grenade or rocket launcher
* added lower sample rate and bit sound loading
* stored the fullbright index out in the colormap lump
* new enforcer pain frames / code
* added alternate animation sequences for textures
+ still running into spikes
+ fix host_framerate
+ drop backpacks when floor removed
bound ammo maxs on weapon pickup as well as ammo
diminish sounds through walls
partical puff and light flash from spike shooters
do a disk icon when caching
proper alias model bbox efrags
allow monster hearing at farther distances
animating texture speed control
slipgate lightning sprite
winking out door in tim6
pusher problem on smotte (climbing out of water on a bmodel)
need to allocate as large a sound cache as possible
wizards die on top of spikes in mid air?
doors shouldn't start move sound if blocked open
don't slide down slopes
cache alias models
-----------------------------------------
John Carmack's .plan for Apr 27, 1996
-----------------------------------------
* alternate texture animation sequences for depressed buttons
* dropped knight and ogre melee damage
* arbitrary texture axis on surfaces through all the utilities
+ remove .modified
+ need to make qbsp do 256 subdivision based on texture axis
+ monsters still hit pathcorners when hostile
+ demon jumps don't do damage
triggering an open door still makes a noise
allow turning off styled lights
QuakeEd texture x size field is clamping at 64 min
faster mouse forward/backward?
is it jumping higher when running?
gun puffs still hit sky
get all visible entities into the low 256 or 128 edicts
force low quality sounds on low memory machines
still have some light edge blobs
-----------------------------------------
John Carmack's .plan for Apr 29, 1996
-----------------------------------------
* fixed light sample determination for all rotations and scales
* pulled wizard shot starts inside bounding box
* fixed demon jump damage
* objects never clip against their owner, and vice versa
* ogre pain frames
* fix pathcorner bug
* don't let monsters see into water
* don't allow dead monsters to be used by triggers
* reset player's gib head to proper bounding box
* gib dogs
* killed avelocity when bounce objects stop
* toss objects drop with plats now
* remove progrestart
* wizard spikes don't fire if dead
* make wizard drop altitude
* better stereo seperation
+ check message triggers
alpha qlumpy still has wrong type numbers
help screen?
auto reload savegame
add pak building to qcc
let doors target other items
guys gibbed by boxes go to no such frame?
is axe view skipping frames?
do player pain frames cause weapon frame skipping?
highlight current weapon in status bar
dynamically light bmodels
swimming motion
explosions need to reveal secrets
-----------------------------------------
John Carmack's .plan for Apr 30, 1996
-----------------------------------------
* clamp health and ammo on level changes
* removed modified field from edicts
* added pause before ogre first attack
* counter triggers don't fire on -1 or less
* added "give" command for testing
* added highlight on active weapon on status bar
* fixed message triggers
* styled light now correctly light alias models
* restart level with entering items
* increased rocket radius damage
* new swimming code
* changed sizedown aspect ration to mimic full screen instead of status
bar
fix progrestart
sound streaming
klook is broken
don't slide on slopes
optimize server code more
get fakeplayer spots working again as monster targets
longer pause for ogre first shot
knights run up ramp on tim7 and stand in air
style light alias models
ogre only fall down when surprise pain
monsters should get angry at each other more often
increment sound channels so items never collide
powerups
vomitus
hell knight
blobs
work on dogs
monsters sim up/down
jump triggers
low grav aiming ogres
throw monsters with explosions
per level-gravity setting
explosion cores
monsters take damage in lava
turn towards killer when dead
slope lookup needs to match stairs
no monster pain frames in ultra-hard mode
monster moves after killing player
disk icon during savegame
smooth center instead of snap center
sometimes multiple death sounds don't start
don't use stdio lib from savegame
"player switches to " messages
info bar in single player: current secrets, kills, level name, etc
load game unpauses
pick up own grenades without explosions?
monsters fight each other after player death
allways switch to picked up weapons
switch to super weapons automatically?
light styles in savegame
prog access uninitialized edict field causes bus error
get 64 bit clean (progs overlay pointers and floats)
allow multiple pack files
-----------------------------------------
John Carmack's .plan for May 01, 1996
-----------------------------------------
* background noise track code
* fixed swimming nan bug
* fixed Sandy's quakeed crahser
* QuakeEd: turn region off when going to new map
* automatic sound decimation based on available memory
* explosion sprite cores
* QuakeEd: don't go to worldspawn
* qcc is now fully recursive
* buttons only pressable by player
+ the explosion cores are still MOVETYPE_TOSS, not MOVETYPE_NONE
+ set gravity on secret level
+ host_framerate cvar is broken
+ shootable buttons
+ wizards on paths go to wrong heights
+ make knights not move when in melee range
+ light styles in savegames
+ QuakeEd: filter world
+ QuakeEd: remove check for filtered brush
fish fly out of water
make door triggers not box sized
check respawning on tim5 armor that drops down
make an end of frame call for each player
allow sprites to have a faked forward Z value? allways?
key alias models
-----------------------------------------
John Carmack's .plan for May 02, 1996
-----------------------------------------
* changed grenade cores to MOVETYPE_NONE
* set gravity in worldspawn
* fixed host_framerate
* stopped sounds before server shutdown
* clear all controls on first client frame
* view.map, viewmodel, viewframe, viewprev, viewnaext commands
* dropped ogre backpack to 2 grenades
* fail soft on video mode memory allocation
* checked console resize in vid mode set, so size messages never get lost
* shootable buttons
+ edge light seams
+ increase max packet size during development?
+ rotate dogs slower?
+ monsters drop to floor after teleport
+ need very obvious sound for sequence completion
+ model viewer inside quake
+ cache alias models, if only to help level to level change speeds
+ cache heap compaction
+ remove bsp->leaktest from QuakeEd
+ dog leaping
+ sight sound on monster use
+ allways switch to picked up weapons
+ blink weapon icons on status bar when picked up
+ climb out of water bug
+ water warp option in all vid modes
+ make qcc 64 bit clean by using doubles instead of floats on 64 bit
archs?
intermission levels
spritegen has funny pathname determination
demo fast forward
seperate impact from damage
combine some of the state flags in both client_t and server_t
ambient light option on bmodels
palette should be reset whenever console goes full creen
should all +- binding have the keyname prepended to avoid the multiple
downs?
seperate mouse scaling for turning and moving
make a "soundshot" option fortesting soundcode in ansi spec makefile?
raise / lower weapons
change weapon after firing last round
flying fish
change entity flags to effects
should host_error try to cleanup aborted servers and clients?
how many hunk allocs are there? save memory by combining?
QuakeEd: filter based on skill levels
check trigger firing when spawned inside one
remove input line completely when not accepting input
data checksums for remote servers
shrink net packet sizes
-----------------------------------------
John Carmack's .plan for May 03, 1996
-----------------------------------------
* reduced rocket refire rate
* removed ogre wait after awake
* added shambler pain finished time
* new caching code
+ staircase rail on tim6 needs to be taller
+ normal server disconnect is reported as a host_error
+ jrbase1 up arrow out of slime is near a step
+ bad changeup to green armor from yellow? level transition?
+ are demo playback speeds correct?
+ stair walking underwater
+ ogre explosions need cores
+ teleporting monsters left a statues
+ shootable buttons don't anger monters properly
+ cycle weapon command
+ pain channel is same as weapon channel
- cut swimming speed some
- go through more get up frames on zombie before failing
+ got stuck in a dropping zombie
+ wizads on paths drop to ground
+ torch models
+ remove keys on use
- different keys on status bar for different levels
+ all items in inventory need to blink on pickup
+ soldiers seeing into water on tim6?
savegames cause enemy items...
different color console fonts
monsters need to hear from farther away
think about patch / update plan
precache_ambient
alternate surface warp
killing ogre while on falling platform left him in the air
better treading water
better search path handling
make a progs / models test directory for kevin
blink key icons on doors
default.cfg allways goes to knwon state
animate key icon on doors when failing to open
-----------------------------------------
John Carmack's .plan for May 04, 1996
-----------------------------------------
* allow "quake f:\quake\id1" for better ISDN development
* dynamic caching of alias models
* fix the surface cache size instead of letting it fluctuate
* explosion cores for boxes and ogre grenades
* transfer monster anger from button press
* fix trigger message printing for secret level
* added notouch flag for triggers
* added nomessage flag for counter triggers
* fixed secret counter triggers that are fired instead of touched
* fixed fired button activator handling
* fixed wizard path following
+ alias model torches...
+ don't spawn triggers for key doors, just use touch
+ make sound channel 0 never match
+ jrbase1: don't put guard inside ammo box, superhealth at end is no good,
bridge delay is wrong, can still fall into slime
+ debug the stdio library problems!
tim5 armor drop on ogre fucks up
remove UPDATE_BINARY and pushmove
make +_ key commands pass source as second parameter
cace console background?
more alert monsters?
demo cycling breaks netgames
automatic treading water
squish to bloody paste on interference
-----------------------------------------
John Carmack's .plan for May 05, 1996
-----------------------------------------
* removed player entity from own entity set
* fixed memory trash bug from not flushing surface cache before
changelevel
* use all memory between the hunks as dynamic cache
+ dump time between levels
+ sticky bsp corner
+ fix wall jump scaling
+ better backpack messages
QuakeEd is doing something over the net on NXPings
bloody face health indicator?
crosshair?
don't pass angles, frame, etc on viewentity
rockets spawn past point blank
Single player names on qsockets are backwards
make dead player still solid to monsters to prevent seeing inside models?
-----------------------------------------
John Carmack's .plan for May 06, 1996
-----------------------------------------
* removed MOUSE1, MOUSE2, MOUSE3, added K_MOUSE1, etc for binding
* check for spurious mouse buttons
* fixed stair climbing underwater
* trigger_onlyregistered
* fixed door sound loopinhg problems
* button configuration
* made monsters stand still at point blank range
* idle view motion
* monsters get mad at each other on any damage
+ cache movement on hunk allocs
+ hi res modes with new code are broken
+ dead guy still firing / pushed into floor
+ fake pause causes problems when map from console
+ zombie shouldn't go into pain frame when down
+ change weapon command
+ is DMA left enambled in sound code?
respawning of items that dropped down with a plat
rocket points block plats
don't clear text on vid mode change
grenade bounced off ogre
bind # option for weird controllers
key sticking
investigate the vis problems
make monsters head for their spawn points after they kill the player
camera follow killer
destroy backpacks when doors close on them
lightning gun model
lightning gun no damage to shambler
does shambler bolt hurt monsters?
-----------------------------------------
John Carmack's .plan for May 07, 1996
-----------------------------------------
* fixed point / point missile interactions
* don't change weapons on pickup in deathmatch
* made vis 30% faster
* fixed falling in floor
* fixed and armor plated the jump out of water code
* fixed jump monkey climbing
* dissalow "kill" when allready dead
* all hell knight actions
* COM_LoadStackFile
* cache movement
* fixed shambler pain frame
+ stair climbing clipping against walls is messed up a bit
+ monsters need to drop to floor after teleport
+ menu01
+ loading on stack causes errors on dos
+ flashing disk icon
+ make caching perfect
+ monsters not fight each other if not seen player yet
+ make lighting gun consume cells
+ falling into floor bug
+ zombie clipping
+ cycling channel allocation
+ no Host_Error: for valid disconnect
+ body que isn't working
shooting rockets at point blank
deathmatch switch to better weapon again
lightning needs to pitch
control animating texture speed
need to cut the update message size
sfx volume option
player only checkbox for doors and triggers
launch lightning from exact spots
finalize HMD interface
repeating grenade bounce on some slopes
jump out of water automatically when swim up to an edge, no need to jump
new ground explosions
auto pitch went into look down for no reason
gib velocity reletive to damage?
-----------------------------------------
John Carmack's .plan for May 08, 1996
-----------------------------------------
* fixed crash on machines with no sound card
* made texmake grab new palette
* removed tent generated soudns from progs
* fixed enemy anger on damage
* sound channel 0 never overrides another sound
* basic blob
* light styles in savegames
* Cash's fix for savegame slowness
* door messages
* fixed door retriggering sound and mixed size letdowns
* changed zombie pain / clipping code
* enforcer backpacks and alt death frames
* trigger monsters play awake sound
* longer ogre aim time
* fixed checkattacks crossing water
* new bit coding on entity updates
+ lightstyles in savegames
+ broke multiple message triggers
+ key door sounds
+ messages on double doors
+ messages on secret doors
+ 3 digit frag bar
+ light edges
+ change weapon command
+ center print messages
+ hell knight slice
+ more wizard death frames
+ underwater noise
make gib entity rotation client side only
ambient level on doors
decouple weapon animations from pain, etc
light bmodels
point blank rockets
look at killer monster
swing weapons down
turn view when axe fighting
grenade destroy bodies
lightning/rocket shooter
swimming frames
teleport warping
teleporter surface warp
blow up corpses
get all visible enttites into low 128 or 256
-----------------------------------------
John Carmack's .plan for May 09, 1996
-----------------------------------------
* fixed door linking / messages / targeting
* remove onground after teleport
* fixed zombie pain and getting up inside players
* hell knights don't anger each other
* added source entity to prog traceline
* fixed point blank rocket bug
* new light sample point determination
+ no host_error on proper disconnect
+ change weapon
+ powerups
+ cell packs
+ fix pausing on menu / console
+ make key doors touch activated instead of triggered
+ lightning gun use ammo
+ word wrap consoke
+ monster jump triggers
- blob spit attack
is turning not happening when in contact with monsters?
client side lightning
lightning gun
enforcer bolt
more pain feedback
setting up the vid mode then changing levels can cause out of memory
give a spike when hit by spikeshooter
monsters burning in slime
hknight throw a spike occasionally when charging
demon jump down from higher
higher demon jumps
enforcers still see underwater
-----------------------------------------
John Carmack's .plan for May 10, 1996
-----------------------------------------
* fixed undead demon bug
* new main menu
* everything splashes on entering water
* bounded ammo on weapon pickup
* added cell models and stats
* Cash's screenshot error check
* Cash's Host_EndGame
* Cash's net packetsize bump
* disc icon during IO
* monster jump triggers
* cache optimizations in edge refresh
+ cycle new allocation numbers
+ make qcc do pack files to a destination
+ dog leaping
+ wizard death frames
+ death popping
+ step up wall clipping
+ crucified zombie
+ nudge weapons with sized down screens
+ 64 bit clean qcc
remove drop to floor?
make Quit go back to demos and reset controls for E3
lots of demos
slime damage for all monsters
pass PVS list to refresh instead of passing efrags
remap stupid entities (item_weapon)
sound for all monsters hitting ground
reset the bonus items in monsters
refix the rocket stuff after demo
-----------------------------------------
John Carmack's .plan for May 11, 1996
-----------------------------------------
* wizard death frames
* dog leaping
* make key doors touches
* increased air time by two seconds
* new keyboard pitch aiming aids
* crucified zombies
* cemter printing
+ tally monster deaths at end instead of counting at kills (monster on
monster)
+ parse \n out of messages
+ status bar feedback
+ klook
weapon dropping
weapon jumping
mess with configuration screen
move all messages into a language qc file
fix frag bar
some side velocity on death
check monster full turn before firing
check exit hang on beta machine
puff+light for spike shooters
tim5 armor fall on ogre bug
view track killer
enforcer bolts
less monster alertness when high or low
blob spawn ceiling
fix coop
-----------------------------------------
John Carmack's .plan for May 12, 1996
-----------------------------------------
* catch overbackoff in movement sliding code
* extended key/value size in utils
* added flame alias model
* \n in epair text
* fixed slope by wall false step in movement code
* hacked around sticky corner problem
* weapon/item flashing on inventory bar
* fixed soundinfo exploded on a no sound card system
* optimized snd paint and transfer
+ change weapon
+ jrstart: no fall in water by medium
+ demo loops
+ player stats on frag bar
+ nail gun alternating
+ cycle edict allocations
+ jrstart: light up black corners
+ clear center print when console drops
axe view tilt
better underwater bubble control
track killer
kick monsters off edges in pain/death
jump landing frames
new explosions
-----------------------------------------
John Carmack's .plan for May 13, 1996
-----------------------------------------
* weapon cuycling command
* trapped bug with maps without textures
* increase key/value size in QuakeEd
* qcc -pak option
* "flechettes" to nails
* added "killtarget" on all SUB_UseTargets
* secret doors do SUB_UseTargets
* secret door touch messages
* cycle entity allocation
* versioning in savegames
* save skill in savegame
* don't set onground when standing on another monster
* fixed loadgame during demo bug
* put falling crunch on voice channel so it overrides pain sound
* fopen demos inside packfile
* automatic ambient sounds
+ bad recursive explobox?
+ ambient sounder tents
+ tab stats
+ center print sequence stuff
+ demo files in packfiles
+ powerups
+ underwater looping sound
+ pass source as parm for +defs
coop start spot
don't accept mouse view when paused
"you got # shells" for item messags
precache_ambient
visbug.map
seperate impact from damage
create pak files with search path?
tim5 drop item on monster bug
texture animation speeds
squash to paste
doubled "clearing memory"
parse /* */ comments in config files
explosion has bad colors in it
proof of purchase
lightning gun
treading water
looking down towards ground bug
deathmatch level transitions
better lava balls
is zbuffer precision maximized?
alias subdivider used wrongly in zoomin mode
permanently shifted view angle on restart
optimized single channel ambient mixing
are 0 volume sounds hanging around?
jrbase3 old bsp explosion
auto jump out of water
demo file version numbers
-----------------------------------------
John Carmack's .plan for May 14, 1996
-----------------------------------------
* fixed recursive barrel death
* new characters
* better ambient name parsing
* con_dpritnf for developer messages
* start messages out of maps
* directional touch triggers
* remove qtest license stuff
* fixed off by one error with ambient leaf calc
* level spawning messages
* clear input line every time console goes up
* moved all non-refresh vid.buffer access into draw.c
* removed alphalumpy -- use qlumpy
* automatic extension and dest file creation for qbsp
* merged id1 and id1_ development hierarchies
* Made QuakeEd projects directory independent
+ raven's bug map
+ static point ambients
+ hell knight smash
+ flameb1 - 11
+ explodes on manually going to bad map
+ play humm1 for teleport textures
+ parms for all + commands to identify source
blood/chunk spray direction from claw attacks
shootable base texture needs an off frame
lint the utilities
cache console background
make all jump trigger targets have a land sound
center menu in high res
slipgate animations
loading plaque, no console between levels
supress input line
fix timedemo with cycling demos
gun jerking
!cl.worldmodel is no longer valid?
mike signon messages stay in notify lines
break off anger at another monster if can't reach in ten seconds
debounce all keys?
center print level messages
still dropping pack in single player?
debug alpha modelgen
-----------------------------------------
John Carmack's .plan for May 15, 1996
-----------------------------------------
* fixed bug with cl.disconnect not clearing worldmodel
* linted and asnified qbsp/light/vis
* changed sbar update model
* powerup colors and icons
* fixed console ] gets messed up with escape/escape
* fixed weapon icon flash going from level to level
* solo console
* blob fly1-4
+ QuakeEd: don't allow retexture in wireframe mode
+ QuakeEd: clear wad on bad load
+ unfork qbsp
+ body que
remove any remaining static limits in qbsp
split client state into level - game - invocation
eyes on status bar for invisibility
accumulate damages so quad dmg shotgun can gib
fix spit.map
optional FOV cull for modem?
super armor number
server send prog checksum to clients
fix hard coded location of status bar numbers in wad
Cash's teleporter issue
fake some angles to hide latency on movement
faster mouse forward/backward
+ QuakeEd: clear brush heights on new level
more edicts
+strafelook
toughen up shambler
no demon/demon blob/blob violence
hell knight anims
dopg death frames
outside level plays all ambient sounds
cl.active isn't good anymore
demo loops crew multiplayer
give for powerups
make clip brush only models work
fix bug_numpoints_on_hp.map
-----------------------------------------
John Carmack's .plan for May 18, 1996
-----------------------------------------
* make func_wall toggle frame on use
* QuakeEd: remove bsp->leaktest
* QuakeEd: filter world
* QuakeEd: remove clicked on regioned brush message
* QuakeEd: don't allow retexture in wireframe mode
* QuakeEd: new brush heights are now snapped to grid
* default .bsp extension to bspinfo
+ enforcer bolts
+ sound for all monsters hitting ground
+ 64 bit clean qcc
+ parms for all + commands to identify source
+ body que isn't working
teleport effect
global sfx volume option
frag bar highlight
permanently shifted view angle on restart
goddamn dos \r text shit in output from NT tools messes up NeXT tools
check for small area polygons and portals
-----------------------------------------
John Carmack's .plan for May 20, 1996
-----------------------------------------
* server sounds parameter specifies soundtrack
* fixed cursor bug when clearing input line on console raise
* fixed problem with e4 ambient (32 bit overflow)
* ambient teleporter sound
* snd_show debugging tool
* fixed underwater grenade sounds
* fixed shambler hold attack until in range
* ogre chainsaw on side swing
* claw meat spray
* f1 toggle help
* no end of level stats on empty levels
* fixed tilted head savegame bug
* don't drop backpack in single player
* wrote unpack utility
+ teleport static sound
+ refix rocket point blank
+ make fire animations non-syncronous
make MAX_PACKNAME define and error check against it
dissable qcache for release?
death/restart from loadgame
should mouse forward/backwards cause an auto-center?
unpackdir
load/save should not take full paths
ambient val on bmodels
dynamic lights on bmodels
finale
fix demo cycle problems
player corner clipping can take too long
option to report any unfilled pixels (jrmed2)
QuakeEd: goto point in command window
monsters shooting you underwater
-----------------------------------------
John Carmack's .plan for May 21, 1996
-----------------------------------------
* passage research
* fixed static sound start with -nosound
* optimized SV_CheckBottom for easy case
* sound optimization
* fixed kill coutners for world kills
* pop blood trails up higher
* key doors need to have touch removed
* added volume cvar to post scale all sound
* underscale sound
+ make hell knight only send one dynamic light out
+ tim6: wordtype typo
+ bmodel pvs issue in tim6
shotgun sound has dead time at end
damage amp in jrbase1 is pointless
easy to use "is drive readahead" test
listen server bugs
menu sounds
weapon offsets are wrong again
damage amplifier bug
check cd removal issues
tighten muzzle flash
phantom nail boxes in tim6
heads get stuck in ceilings sometimes
is lightning owner set properly? sandy got stuck using the lighting gun
-----------------------------------------
John Carmack's .plan for May 22, 1996
-----------------------------------------
* complete mouse configuration: m_pitch, m_yaw, m_forward, m_side
* m_filter sample smoothing option
* split non clearable parts of client_state_t into client_static_t
* clear items at level exit
* make hell knight only send one dynamic light out
* fix klook
* smooth centerview
* properly handle multiple inputs on a button, got rid of +forward2, +attack2
* exit game text screen
* gave name to unconnected clients
+ make torch animation client side!!!
is near Z clipping of alias models a bad idea?
allow multiple packfiles to override one another
better ideal pitch on slopes
mouse pitch/strafe combo
look at redbook audio
restart shouldn't drop console.
check for reloading the same level.
noambient flag for low memory systems
check zone memory allocation for bindings
differentiate reconnect
allow up/down in water
swimming frames
print bindings option
more edge friction?
reletive volumes for each ambient sound
explicit bindings for key up / key down?
teamplay totals
level name in solo scoreboard
-----------------------------------------
John Carmack's .plan for May 23, 1996
-----------------------------------------
* fixed body que
* Michael's client side alias animations
* Cash's fix for net connects
* fixed bobbing pvs problem
* fixed on-edge water grey problem
* fixed console keys generating double release events
* fix new pitching recenter
* changed powerup effect to dim light
partial screen updates
bloody face?
player induced meat chunks
seperate soudntrack files into seperate download?
check sighting into water
scroll water textures?
accumulate shotgun damage for powerup gibbing
override help/sell and finale in registered pak
victory music
append sample rate to music filenames to match best
rename jrstart to start
rename the last I_ functions to sys_
cache textures?
dim out screen when menus are up?
comment line on menu screens
del in save field
scan for all id1*.pak at startup
reclaim entity string space after spawning
change mins/maxs from floats to shorts in bspfile
cut vertex numbers to shorts?
marksurfaces to shorts?
remove view model in weird FOVs
ammo boxes not vanishing after reload
change dynamic light attenuations for tighter spheres
prog def for shareware only version?
-----------------------------------------
John Carmack's .plan for May 24, 1996
-----------------------------------------
* added lookspring back
* fixed client PVS for checkclient to be from viewofs
* fixed no sighting underwater
* no tarbaby damage to each other
* blob explosion core
* cleared prog stack on prog error, so recursion pverflow doesn't give
error
* fixed shalrath pain frame bug
* settriggermodel() now does angle, movetype, and solid init
* reduced client to server packet size
* trigger_changelevel fires targets
* end of level stuff
write a multi-platfirn C shell script for makefiles
360 jump trigger
shootable triggers?
take away all screen effects at completion
intermission code
more monster awareness to nearby carnage
reduce monster accuracy with invisibility
high res centerings
remove counters if cheating
rename cheat codes
generalize button0 button1 to button 0 - button 7
dynamically reconfigure listen clients
weapon movement with screen aspect is broken
option to not pause the game while console is down?
longer underwater time in easy skill?
set registered flag
menu sound (pause issue?)
remove shootable invisible triggers?
go over all the prog code
gamma in options menu
allow attack to respawn as well as jump?
put end screen in gfx
killed by box restart bug
video mode set bug
finish invisibility in deathmatch
intermission cameras
QuakeEd view pos button
save polygons by filling inside of sky volumes
load / save game
monster on plat issues
fish
shalrath
good lightning gun
escape twice to remove console
centerspeed not used?
optional center on mlook release
need bigger surface cache in higher res modes
run direct from cd working path option
cull distant sounds / partical effects on the server side?
change vector view_ofs to scalar viewheight
is the view model pitching backwards?
non-thrashing precache?
rename host_client to sv_client
check surface cache size performance figures
-----------------------------------------
John Carmack's .plan for May 25, 1996
-----------------------------------------
* fixed savegame / phantom box bug
* fixed underwater
* allow monsters to sight other just-damaged monsters as clients
* fixed lookspring
+ secrets and kills are backwards
tighten dlight radius/attenuation
copy center prints to console
the portal bug
cue music on completion
-----------------------------------------
John Carmack's .plan for May 26, 1996
-----------------------------------------
* flipped secret and kill plaques
* finish solo scoreboard
* finale
* optimized CL_CalcAmbientSounds
* optimized ResampleSfx
* reduced index sizes in .bsp files
* flood filled away outside sky fragments in bsp files
+ fix cache dir
demo loop issues
modify qbsp to allow entities only option to generate new triggers
randomize intermission spots
conditional bars in jrstart?
teleport sprite
fish
registration key
light bmodels
different quad damage effect
slipgate animations
lightning gun
general "no move" flag for player server C code
scaled texture lighting still has bugs
-----------------------------------------
John Carmack's .plan for May 27, 1996
-----------------------------------------
* moved ambient sound calculations to vis program
* moved file caching option to host_parms
* removed keybindings[1]
* added ambush flag to monsters
* made precache_* functions also return their parameter
* added precache_*2 functions for the registered version seperation
* modified qcc for -pak / -pak2 option to build shareware / registered
* random intermission_info selection
* automatic configuration saving
* angle 0 jump triggers
* fixed solo map name centering
artifact blinking before removal
deathmatch invisibility
blinking cursor for configuration screen
automatic config saving
sum quad damage for gibbing
shrink cl_entities to be cl_update_entities + cl_baselines
tim5 armor fall through bug
radius damage opening secret doors
only surpress autocenter when not on ground, not water check
clearing memory twice, allow baselevel to move if not connected
-----------------------------------------
John Carmack's .plan for May 28, 1996
-----------------------------------------
* QuakeEd: forbid dragging position / direction when textured
* constrained all file writes to the game directory to close security
holes
* allow multiple pak files intermixed with directory trees in the search
path
* full specification with: -basedir, -game, -cachepath, or -path
* moved demo list to cls, fixed efrag bug on second demo playback
+ bring up menu on any input during demos
+ do "loading" plaque from demo to demo
+ demo transition problem
+ automatic pak grabbing frame game dir
+ saving while in a trigger field kills it?
bump edicts again
acknowledge all - command line parms as they are accepted?
QuakeEd: don't explode if /qcache isn't present
qcc option to re-bsp models and maps
shambler lightning needs to be exactly positioned
joysticks
keybinding saving isn't right
claim all args from command line, or print warnings?
-----------------------------------------
John Carmack's .plan for May 29, 1996
-----------------------------------------
* any key in demos brings up menu
* demo transitions bring up loading plaque
* added timeout for loading plaque
* forced console away with loading plaque
* regular restart uses loading plaque
* disabled triggers now work properly after a savegame
* cleared edict before parsing to fix door beeping bug
* testing if(string) in qcc is not reliable, change to if (string != "")
* fixed loadgame cursor problem after saving a typed in name
* fixed dos savegame crash bug
* automatic pak0.pak pak1.pak etc loading from game directories
back to start after finishing each episode
episode finished items
ogre jumping frames
load bmodels by lumps for less memory damage?
does quake.qpr have to be in the root of the development directory?
crosshair option
wait on jump triggers?
drowning warning
cmd line load
reformat text on vid mode change
deathmatch level crossing
nightmare mode
death restart needs loading plaque
map name on savegame comment line
regular restart after a load game doesn't work
bound load pointer
swing up weapons
lightning gun
safe edict allocation failure
remove view model in wide FOV modes
give 9 faults
-----------------------------------------
John Carmack's .plan for May 30, 1996
-----------------------------------------
* automatic setting of the registered cvar
* demos play from pak file
* delay SCR_EndLoadingPlaque until after cache report
* added -minmemory parm for testing entry case
* stopped demo loop on demo load error
* stopped demo loop on connect and map
* zbuffer torch dropout bug fixed
* changed all remaining I_ functions to Sys_
* chase network issues
* colored light gel demo
remove bspleaktest from quake.qpr
can trigger_changelevel stuff two commands sometimes?
remove powerup shifts before intermission
gib code for the big monsters
give wizards (hknights?) need a muzzle flash
last_damage on zombie for cumulative gibs
new explosion
check min memory with large number of clients
better treading water
escape at a fulls creen console to restart demos?
allways run checkbox in options
instructions on configuration screen
fish attack
fish not leave water
finish invisibility
conditional registration message in jrstart
monsters die in slime
longer message triggers in jrstart
take dogs by ramp out in jrbase1
easier jump out of water
remove jsilly rbase1 bonus items
smotte light at start
loadgame from pulldown menu
amlev98 shambler needs red armor on easy level
for release: don't forget to remove "developer 1" cvar and bound cheats
-----------------------------------------
John Carmack's .plan for May 31, 1996
-----------------------------------------
* based load plaque removal on framecount instead of updates
* fixed crash bug with zerod notify lines
* load / save game menus converted to slots instead of file names
* end1 / end2 shareware / registered end screens
* triggered monsters no longer awake on invisible or notarget players
* broke out default.cfg from quake.rc
* fixed five way counter trigger message
* fixed push triggers to be frame rate independent
* translated spaces to _ in savegame comment to make stdio happy
* escape in keygrab now justs turns it off
* protected against recursive loop when Con_Printf inside
SCR_UpdateScreen
is model name caching being done correctly for the flushed world model
name?
centerprint message delay times
fade screen behind menus
gib big monsters
cl itemgettimes aren't saved. do they need to be?
weapon nudges on different view sizes
slime ambient sounds
center everything in higher res modes
got stuck by explosion near zombies
return to jrstart stuff
amtest98 needs new teleport textures
reset defaults option
reset to defaults option
track down load failed comments
why do menu sounds ometimes not work?
combo look/strafe
fix lightning
push trigger directionality wierdness
-----------------------------------------
John Carmack's .plan for Jun 01, 1996
-----------------------------------------
* tracked loadgame problem to stack overflow
* removed console cursor if key_dest != key_console
* ignore all autorepeat keys except backspace
* removed the extra unit of padding around bmodels that caused them to
clip
* build entity list for refresh instead of pushing efrags
* fixed cells not carrying from level to level
* nightmare skill item spawning is fixed
* old deathmatch rules by setting deathmatch 2
* options menu sliders
* moved gamma correction out of vid systems into view
+ shambler lightning needs pitch
- stop demo when menu comes up?
+ nightmare skill
+ integrate spritegen with
+ lookspring checkbox
+ make sure all menu options get saved
+ clear notify lines when in menu
increase surface cache size?
do knight swords do meat chunks?
centerspeed slider
-----------------------------------------
John Carmack's .plan for Jun 02, 1996
-----------------------------------------
* fish don't leave water
* merged new CD code
* shambler lightning pitch correct
* cut number of cells in half
* beam weapon entities only on client side
* fixed sliding death/no retart bug
* muzzle flash for wizards
+ regrab gold/silver keys on sbar
+ accumulate damage over a frame
+ dissalow more than two bindings to the same action
flymonster in wall test
jump cause center?
non-linear mouse movement reponse?
use 3 frames of gib3
start using pcx for all utilities
use slight for shambler setup
replace all item_weapon
lava damage on monsters
transition to intermission
warning sound on about to drown
don't end demo when player is in menu
-----------------------------------------
John Carmack's .plan for Jun 03, 1996
-----------------------------------------
* gate_boss, gate_episode
* item_sigil
* fixed demo transition forcing console and menu away
* setup for server transitions with connected clients
+ grab new explosions
+ console screen isn't coming down in deathmatch
+ non-pickup nails again
+ console cursor
+ check tim21
+ coop player starts
+ flip mouse y as a checkbox option
monsters aren't telefragged?
aim lighting gun up/down
connect while demo is running fucks up
sound for quad damage attack
accumulate damage
shoot flying gibs?
add new gibs
deathmatch intermission screen
talk message buffer doesn't erase with sized down screen
-----------------------------------------
John Carmack's .plan for Jun 04, 1996
-----------------------------------------
* allowed host_framerate to explicitly exceed the normal bounds
* reduced cls.state to ca_dedicated, ca_disconnected, ca_connected
* got consistant on when memory is dumped -- only at start, not end
* increaded MAX_VALUE to 1024 for longer text messages in entities
+ dlight bmodels
+ escape with console down brings up menu in addition to removing the
console
+ s_init should not be called on dedicated server
+ sound channel droppoff when lots going? combine torch sounds?
+ only allow two bindings to an action from menu
+ delete in options menu to unbind all
+ carry serverflags from level to level
+ make all items fire triggers
+ coop respawn
check dedicated, listen, loadgame, recorddemo, restarting, timedemo, and
demoloops with new server setup
make hit ground sound entity reletive
filling sky volumes causes rockets to explode on sky. fix somehow
make all teleporters have an "inactive" flag so an event makes them active
QuakeEd: top trace for shearing is at 2048, increase to 8192
remove relight for quad damage
remove leading K_ from keynames
player swimming frames
flash ammo counts after picking up bonus items?
shalrath doesn't attack zombies?
-----------------------------------------
John Carmack's .plan for Jun 05, 1996
-----------------------------------------
* coop is mostly working
* made snd_show give a total count
* combined static sounds of the same type
* demo record is now reord [demoname] [map]
* intermission takes all players in coop
* all players are invisible for coop intermission
* coop telefrag
* dead players are respawned on level change in coop
* don't bound clinet only systems to sys_ticrate
* fixed bug with monsters not sighting players after some level changes
* keys aren't picked up in coop
* moved colon on intermission
* cleared effects on player respawn
* fixed players 2+ not saving stuff on entering level
* coop colors follow from level to level
+ backspace / delete to remove bindings in options screen
+ coop quit doesn't send disconnect
+ super spike: s_spike
deathmatch weapon switching rules
blue sparks for lightning
check for splash before playing thump
center or scale menus in hi-res
change "gamma" to something else because of name clash with new djgpp
angle jump at demo start
coop should use solo scoreboard
alphabetize cvars / commands for tab completion
look for "backup past 0"
fix "gravity changed" message
powerup blinking
frozen demons?
timer on monsters mad at other players
-----------------------------------------
John Carmack's .plan for Jun 06, 1996
-----------------------------------------
* carry serverflags from level to level
* items fire targets when picked up
* different spikes for super, knight, and wiz
* removed requirement that triggers have a target
* removed all references to update_type
* lightning gun underwater explosion removes currentammo
* fixed bug in fat pvs that caused entire world to show on some frames
*.5 trigger only teleporters
* delete key bindings and prohibit tripple bindings
* blured savegame line
* don't allow cmd buffer sizebuf overflow
* behind menu fade
* seperate cursors for console and menu
* changed entity spawning to spawn-as-loaded
* move light entity hack to pogs
* draw sigils on status bar
+ remove self.running?
+ portal creation bug
+ make lightning gun more damaging again
+ sigil things on status bar
+ signon1 when at 1
+ weapon nudges
+ player faces
+ put talk.wav back in
+ wait before allowing intermission exit
+ teleport ambient sound?
quad damage liughtning explosion
menu4.wav for slider
make a generic signon stream and subdivide svs.signon more
velocity through levels?
all projectiles need a t ent
finish axe up
pickup weapons while firing bug
tick sound when pressing an unbound key?
escape back through menus
loadgame removes enemy state?
deathmatch intermission
texture animation speeds variable
use different exiting water sound
-----------------------------------------
John Carmack's .plan for Jun 07, 1996
-----------------------------------------
* three levels of gib velocity
* accumulate damage to allow gibbing with shotguns when powered up
* radius damage opens secrets
* get Michael's bmodel lighting code
* moved notarget check into progs code
* removed self.running from progs
* fixed weapon fov nudges, moved super shotgun
* status bar faces
* fixed single player loading plaques
* auto center if on bmodels
* sound and cd audio have big banners if not configured
* grunt muzzle flash sync with attack
* darkened blood
* dog head
* moved sound and cd init into non dedicated startup
* moved message printing to SUB_UseTargets (buttons get messages)
* moved delay to SUB_UseTargets so all activations can delay
* trigger_relay entity
* clear notify and center print when in menu
* added svs.changelevel_issued to prevent possible double changelevel
* changed NaN velocity / origin checks to warnings with clear to 0
+ take away stuff in jrstart
+ player head gib now has wrong view ofs
+ the tim5 bug
+ exactly position enforcer bolt launch
+ status bar faces
+ save game with powerups
+ monster target finding with prespawning is messed up
make activator a parm to use functions instead of a global
write a tool to search all maps for boolean queries on key/value pairs
reformat console with vid changes
start past skill select when returning to jrstart
connect to bad map, then to a good map gives no free clients
lava damage for monsters
shambler hand lightning
fade to intermission?
meat spray from axe attack
gamma prob at intermissions
make a model fullbright flag for torches
Add -safe for nosound, vidmode 0, exec default -nocdaudio, -nonet
kill zombies with axe
teamplay totals
doesn't auto center if head out of water
easier jump out of water
can't record net demos any more
solo status bar in dark font
key config names in dark font
print savegame comment line on save screen
are you sure? for quit
spawn explosion for lightning gun pop
invis face, invis + invuln face
monsters aren't allways facing target when firing
make the message prog functions take a destination: broadcast, static,
single
-----------------------------------------
John Carmack's .plan for Jun 08, 1996
-----------------------------------------
* teleport triggers UseTqargets
* fixed once triggers
* cleared static sounds between levels
* right justify options menu text
* cells became rockets on level cross
* changed teleport humm attenuation
* lookspring checkbox
* lookstrafe checkbox
* mouse y invert checkbox
* allways run checkbox
* made fov not a saved cvar
* wizspike and knightspike t ents
* take away stuff after completing an episode
* end of episode text
* head gib eye in floor bug
* blink cursor after messageboxes
+ everyone goes through the return teleporter in episode 4 jrstart
+ is bgm volume not working for cd?
+ make zoom a default configuration?
quad damage shotgun doesn't gib zombies
bonus item flashes
no menu sounds when disconnected
teleport ambient sound is no good
auto center when treading water
monster drowning
blaster mismatch warning is misformatted
axe kill zombies
new game loading plaque not showing
change zombie knock down code
reformat console on vid change
grenade launcher kicks too much
surface cache size in weird aspect ratio modes
prevent vid from allocing a big enough buffer to cause the next load to
fail
deathmatch invis eyes
monster reduced accuracy during invis
push for red super spikes
monster death bubbles underwater?
change name back to quad damage
disconnect from netgame isn't being sent
remove coop cheating
centerprint time reletive to length?
centerprint "entering ..." on level entry
-----------------------------------------
John Carmack's .plan for Jun 09, 1996
-----------------------------------------
* boss damage only by lightning
* enforcer bolt positioning
* all gibs centered
* lava splash
* boss stop throwing when player is dead
* fixed cl_backspeed for always run checkbox
* fixed double talk sound on triggers
* retain options menu cursor pos
* nightmare mode: no attack_finished limits, and five second pain interval
* triggered door message bug
* weeded out the unused entities in misc
* consolidated the four remaining sprite scripts into a single file
* modified sprgen to allow multiple grabs in a single file
* grabbed new explosions
+ level 1 gib needs work
+ is clipping detection for asm sound mixing working right?
+ items falling through other items bug
+ item on moving object respawn bug
+ models aren't checking dlight decay time
+ fix drop.map
+ status active/max is reversed
coop against boss
make qlumpy parse qc files as source?
combine particals and temp entities
fix gib rules
slime burn down
check edict return type
flash weapon icons
-----------------------------------------
John Carmack's .plan for Jun 10, 1996
-----------------------------------------
* more velocity on level 1 gib
* no pain sounds when invincible
* reduced lava coor shift
* deterministically spawn items on bmodels
* objects on blocked train no longer fall through
* hell knights need muzzle flash
* fixed flame offsets
* three levels of screen update
* erase exact lines of centerprint
* shareware non-finale transitions
* crash in _SV_TouchLinks
* escape puts away console
* laodgame from demo
* backpacks in coop
* floating gibs
* made beta3 CDs
"restart" in coop game drops to dos
word wrapping needs to count extra notify lines
telefrag isn't working sometimes
put a save plaque up when game is saving
demon died on top of player, left in air when moved
door sounds are broken
registered message shouldn't print at all
menu hotkeys should return to game on escape, not main menu
quickload shouldn't drop to console
quit/end game confirmation
split door is playing talk wav
sound looping pop
weapon frames should be decoupled from pain frames
door touch
take the k_ out of configs
velocity between levels
fish
shub
display key bindings command
deathmatch intermission
door messages
zombie statues
constant look down pitch bug is multi player related
permanent head tilt in coop mode
endscreen
connect while in demo is a bug
secret counts aren't broadcast
half dead player
frozen demon when dropped on player head
swimming frames
map e1m1 from a coop game gives loop qsocket used
all player model axe frames
sw version shouldn't allow path changes unless pop
-----------------------------------------
John Carmack's .plan for Jun 11, 1996
-----------------------------------------
* fixed the vis portalization order problem!
* allow attack to respawn as well as jump
* fixed faulting
* removed sprites directory and all references
* new qcc code for damage, should fix statue monsters
* new zombie pain code
hang on laptop sound init
boss needs to scan for coop players
centerprint powerup messages
change origin offset on fish for more fin
fish move fail on tcross surface should still allow xy move
fish should pull treading people underwater
make good spot for weapon change rules
8 bit sb output isn't using volume scale
slider sounds (esp for sound volume)
is e1m1 nailgun a trigger, or is it duplicated?
play sound or something after intermission will allow exit?
systems that translate blit need to update entire screen on palette
changes
-----------------------------------------
John Carmack's .plan for Jun 12, 1996
-----------------------------------------
* limit sound buffers to 4k, add extra mix points and mix full buffer
* fixed only registered trigger messages
* boss shouldn't spawn in deathmatch
* cleared move commands across level transitions
* 8 bit sound output volume scaling
* key configuration didn't work for changeweapon "impulse 10"
* options menu now resets string values of cvars
* talk drawing properly updates with sized down screen
* clear net connection ques on new connection
* f5 cheat or's in items
* removed error for unconnected buttons
* power up status bar faces
* armor number
* increased surface cache size
* end of game text
* menu up/down wrapping
* Sandy's menu suggestions
* jrstart alternate start point if returning
* play track 3 on intermission
* drop backpacks in coop
* got deathmatch invisibility working
clip into sky problem
demon smashed a player stuck into a sloping wall
no cheat in coop
automatic listen 2
talk message printing code
door sounds
check fly/float move code
check weapon changing while firing
check messagemode bound keys
"restart" on a client of a net game crashes
serverflag only trigger
gib head positioning
ability to specify cd track on demo recording
coop use solo scoreboard
two presses to exit intermission
deathmatch intermission screen
harder nightmare
name and color should be saved as cvars
undead players? fixed with new death code?
QuakeEd: trigger texture setting should be automatic
add colors to dm scoreboard
forward changelevel commands?
good POP
add "serial/modem configuration" menu
-----------------------------------------
John Carmack's .plan for Jun 13, 1996
-----------------------------------------
* slider sounds in menu
* trigger counter's don't require targets now
* fixed blaster mismatch warning message
* net start menu
* enter on full key config clears
* ogre gib
* quad damage face
* quad damage powerup cheat
* all gib heads
* removed K_ from key names
* bind with no parms reports the current value
can't give
try partical effect for teleporters
remove forward to server commands, use impulse for all?
old rule deathmatch option in menu
head in wall again
no cheat in coop
connect from demo
checksum pak directory for shareware version
shambler, wizard, ogre, voor heads
allow arrow key repeats?
item respawn on bmodel bug
pitch jerk at startup sometimes.
stuck heads bug
"no messages" option
axe weaponframe switch problem
make "dropped datagram" a developer only printf
player thump for ogre falling?
centerprint entering message
variable ceterprint wait time
double nightmare attacks
disallow new game during multiplayer?
no player crush to point
sinking into plats
coop through episode end
allow typing in savegame comment
fix talk message display code
keep commands in doubly linked MRU list to avoid linear scan?
multiplayer end of episodes hang?
centerview doesn't work in water
make sure all menu vars are in default.cfg
-----------------------------------------
John Carmack's .plan for Jun 14, 1996
-----------------------------------------
* removed all use of ai_dead in progs
* fixed nailgun fire order
* fixed double press to exit intermission
* fixed sliding dead player bug
* fix bonus item respawn on dropping plat
* zombies don't go down again if already on the way down
* fixed ambient sound looping problem
* show looping sounds lin soundlist
* combined rocket impact and explositon damage for reliable gibs
* fixed stuttering on dropped packets
* set teleport death size properly
easier gib from rockets for players
check telefrag for multi players stuck
is sound buffer not being properly cleared?
menu sounds
dissallow some server forward messages
internet reliable messages getting backed up?
check for hard packet overflows
flash items on status bar
stop using gfx.wad?
pickup rules
teleport into monsters bug
change away from axe bug
-----------------------------------------
John Carmack's .plan for Jun 15, 1996
-----------------------------------------
* handle escape special to prevent unbinding, and pass through to menus
* allow attack to restart as well as jump
* copied more cmd_argv through recursion
* boss lead player in hard mode
* removed blaster warnings on v4 cards
* blinking icons
* protect all cvars and commands that could be deathmatch cheats
* mvoed squish obit before fell to death
* fixed static sound false combining bug
* randomize axe swings
* never go into pain frames during weaponframes
* combined axe damage frames
* prevent bmodel teleport
* registered trigger only accepts players
* fixed activated door beeping
* fixed linked door messages
player crushed on e1m2 bug
make enter play slide noise on menu item adjustments
no head bobbing when dead
gib head look around
sink into rising plats
hack check for stuck
no such frame for gibbed dog head
bmodel dissapearing problem in e4m6
gib by damage isntead of negative health
option for listen servers to send out less packets?
crank menu3 sound up in volume
teleported monster flag
fill sky volumes differently?
shambler lightning hand frames
top line of console is not clipped correctly
door sounds / messages
weapon change rules
no gravity when in solid volume
double shots for monsters in nightmare mode
use multiple help screens
swim bubbles on forard move as well as jump
-----------------------------------------
John Carmack's .plan for Jun 16, 1996
-----------------------------------------
* removed hard coded number order for sbar
* moved palette and colormap to discrete lumps
* all help screens
* delay enter sound until after menu frame drawn
* delated recursive spawn death
* changed sound override rules to prevent monsters from eclipsing player
snds
* delay monster use reaction to guarantee sound after teleport
* added reliable_datagram
* fixed powerup respawning
* clear all of sv in spawn server
* heads bounce from telefrag
* fixed sizing bug with telefragging
* fixed force retouch bug
* bodyque objects get movetypes
player lightning source 16+ higher
check dead player coop transition
talk message needs to clear background
dissallow save from intermission
different default player colors?
kick clients off if unsent message gets too big
powerup flashes
demon and shambler rethink between claws
make boss face direction of throw
deathmatch weapon switching
double shots for monsters in nightmare mode
hot key menus should return to game with a single escape
item respawn with no pickup problem
floating heads
view center snapping
deathmatch intermission screen
directory checks
no frame bug with eyes
head floating in air
reliable broadcast
no exit option for persistant deathmatch servers
statue monsters
change pak files to pakS0 pakR0
triggered teleporters aren't working reliably
play sound when menu goes away
local sounds die sometimes
teleport effect
sounds on locked doors
pops at end of sound buffer: not mixing entire thing?
exp.tri spawn explosion frame
func_episodegate print messages
teleporter articles
fix "load failed" message
coop savegames
lightning gun on secret doors.
explain "kill" in manual
explain dedicated servers for large coop games in manual
-----------------------------------------
John Carmack's .plan for Jun 17, 1996
-----------------------------------------
* lighttning gun always activates secrets now
* fixed coop level to level bug when player is dead
* added final character animations
* only server can changelevel or restart
* fixed scoreboard color bars
* fixed spawn death
* changelevel from client faulted
* pop verification
* pak crc
* min shells on level transition
* voor spike
* coop use solo scoreboard
* teleport particals
* menu sounds
* frag.map
*? view center snapping
* scoreboard flags in higher res
* dm weapon pickup rules
* easier jump out of water
* cd track change at finale boss
* changed center print position for very long text messages
* force cd track on demos
+ -14 cells????
new sky filling
client enter time
end of e4 text crash
deathmatch game exits should stay at intermission
multiplayer boss ai
axe impact sound
rock ogre
stuck spawn
shambler recheck
coop savegames
talk message drawing
broke teleport sounds
stuck problem
water stuck
different pak naming structure
combine mo strip build
-----------------------------------------
John Carmack's .plan for Jun 18, 1996
-----------------------------------------
* fixed cells dropping below 0 with new lightning gun animation
* tighteneed teleport particals
* big hack to catch rare stuck occasions
* prevent save in intermission
* unbindall command
* fixed strip makefile
solo scoreboard not centered
jump out window while looking down looses auto-centering?
drop zombie, then crush it = unkillable zombie
changelevel with a dead server can take away shotgun/axe
knight first damage is too early
-----------------------------------------
John Carmack's .plan for Jun 19, 1996
-----------------------------------------
* fixed "no such frame" invis bug
* coop target tradeoff
* boss target tradeoff
* moved knight damage frames by one state each
* cl_signon messages are now developer only
* changed hell knight hit sound to sw data
* allow onolyregistered triggers to be used as killtriggers
* fencepost bug on name lengths
* added S_ExtraUpdate to Draw_FadeScreen
* ignore nointermission in deathmatch
* notifications for quit and new game
* created 0.9 SW RC distribution
tilde key doesn't type (unbindall)
monster counting bug
net can fuck up with a join
don't let demos change while in a menu
menu sounds still don't work in all cases
play menu sound on escape from menu
stuck spawn
disallow angle change during demo
nail messages say "fell to death"
rewrite buggy centering code
rewrite buggy point squashing code
color coded backpacks
no finale when deathmatching
fish
credits
-----------------------------------------
John Carmack's .plan for Jun 20, 1996
-----------------------------------------
(Shareware release, june 20th missing)
* fish do damage
* fixed lightning prog crash after killing boss
* added ctrl-alt-del check in key_event
* found problem with saving com port enabled in config files
* drop fish hp
* added samelevel flag for persistant servers
* auto advance to next level in ruled deathmatch
* deathmatch rankings show for 5 sec minimum
new game should bring up loading plaque
rewrite buggy screen management
big t dest
pain field on shub
stand in in center
turn off messages option
-----------------------------------------
John Carmack's .plan for Jul 01, 1996
-----------------------------------------
Quake is out. Finally.
I think the end product is damn good, and I am proud of my 18+ months of sweating over it. I hope lots of you enjoy the shareware levels enough to register :-)
While there will probably not be any about-faces in technical decisions, Quake will definately be growing and expanding over the next year or so as we gauge the user community reaction and release updates and the inevitable sequal.
Some work that will be going on in the near future:
Fixing any serious problems with Quake 0.91.
Release of tools and source code for user prog coding and level building.
Writing a new map editor for win/NT + open GL Porting Quake to native win32 + DirectX
Porting Quake on the metal to some 3D accelerator boards.
Writing a proper full radiosity replacement for the light utility.
Reasonable user suggested improvements (keep it in the discussion groups please, I get enough mail as it is)
Speed enhancements in the polygon model path.
Adding support for various gadgets -- HMDs, controllers, maybe other sound cards, etc.
And of course... The Next Generation Technology. (no, I won't give you a code name to talk about!)
Vacation? I don't need a vacation!
John Carmack Id Software
-----------------------------------------
John Carmack's .plan for Aug 02, 1996
-----------------------------------------
Here is The New Plan:
I copied off the quake codebase and set about doing some major improvements. The old v1.01 codebase will still be updated to fix bugs with the current version, but I didn't want to hold back from fixing things properly even if it involves some major changes.
I am focusing on the internet play aspect of the game. While I can lay out a grand clean-sheet-of-paper design, I have chosen to pursue something of a limited enough scope that I can expect to start testing it around the end of the month (august). I still have my grand plans for the future, but I want to get some stuff going NOW.
QuakeWorld.
The code I am developing right now is EXCLUSIVELY for internet play. It will be rolled back into the single player game sometime along the road to Quake 2 (or whatever it turns out to be called), but the experimental QuakeWorld release will consist of seperate programs for the client and the server. They will use the same data as the current registered quake, so the only thing that will be distributed is new executables (they will peacefully coexist with current quake).
There will be a single master server running here at id. Whenever anyone starts up a server, it will register itself with the master server, and whenever a client wants to start a game, it will inquire with the master to find out which servers are available.
Users will have a persistant account, and all frags on the entire internet will be logged. I want us to be able to give a global ranking order of everyone playing the game. You should be able to say, "I am one of the ten best QuakeWorld players in existance", and have the record to back it up. There are all sorts of other cool stats that we could mine out of the data: greatest frags/minute, longest uninterrupted quake game, cruelest to newbies, etc, etc.
For the time being, this is just my pet research project. The new exes will only work with registered Quake, so I can justify it as a registration incentive (don't pirate!).
If it looks feasable, I would like to see internet focused gaming become a justifiable biz direction for us. Its definately cool, but it is uncertain if people can actually make money at it. My halfway thought out proposal for a biz plan is that we let anyone play the game as an anonymous newbie to see if they like it, but to get their name registered and get on the ranking list, they need to pay $10 or so. Newbies would be automatically kicked from servers if a paying customer wants to get on. Sound reasonable?
Technical improvements.
The game physics is being reworked to make it faster and more uniform. Currently, a p90 dedicated server is about 50% loaded with eight players. The new network code causes a higher cpu load, so I am trying to at least overbalance that, and maybe make a little headway. A single p6-200 system should be able to run around ten simultanious eight player servers. Multiple servers running on a single machine will work a lot better with the master server automatically dealing with different port adresses behind the client's back.
A couple subtle features are actually going away. The automatic view tilting on slopes and stairs is buggy in v1.01, and over a couple hundred millisecond latancy connection, it doesn't usually start tilting until you are allready on a different surface, so I just ripped it out entirely. A few other non-crucial game behaviors are also being cut in the interest of making the physics easier to match on the client side.
I'm going to do a good chat mode.
Servers will have good access control lists. If somebody manages to piss off the entire community, we could even ban them at the master server.
The big difference is in the net code. While I can remember and justify all of my decisions about networking from DOOM through Quake, the bottom line is that I was working with the wrong basic assumptions for doing a good internet game. My original design was targeted at <200ms connection latencies. People that have a digital connection to the internet through a good provider get a pretty good game experience. Unfortunately, 99% of the world gets on with a slip or ppp connection over a modem, often through a crappy overcrowded ISP. This gives 300+ ms latencies, minimum. Client. User's modem. ISP's modem. Server. ISP's modem. User's modem. Client. God, that sucks.
Ok, I made a bad call. I have a T1 to my house, so I just wasn't familliar with PPP life. I'm adressing it now.
The first move was to scrap the current net code. It was based on a reliable stream as its original primitive (way back in qtest), then was retrofited to have an unreliable sideband to make internet play feasable. It was a big mess, so I took it out and shot it. The new code has the unreliable packet as its basic primitive, and all the complexities that entails is now visible to the main code instead of hidden under the net api. This is A Good Thing. Goodbye phantom unconnected players, messages not getting through, etc.
The next move was a straightforward attack on latency. The communications channel is not the only thing that contributes to a latent response, and there was some good ground to improve on.
In a perfect environment, the instant you provided any input (pressed a key, moved a mouse, etc) you would have feedback on the screen (or speaker) from the action.
In the real world, even single player games have latency.
A typical game loop goes something like: Read user input. Simulate the world. Render a new graphics scene. Repeat.
If the game is running 15 frames a second, that is 66 ms each frame. The user input will arive at a random point in the frame, so it will be an average of 33 ms before the input is even looked at. The input is then read, and 66 more ms pass before the result is actually displayed to the user, for a total of nearly 100 ms of latency, right on your desktop. (you can even count another 8 ms or so for raster refresh if you want to get picky).
The best way to adress that latency is to just make the game run faster if possible. If the screen was sized down so that the game ran 25 fps, the latency would be down to 60ms. There are a few other things that can be done to shave a bit more off, like short circuiting some late braeking information (like view angles) directly into the refresh stage, bypassing the simulation stage.
The bearing that this all has on net play, aside from setting an upper limit on performance, is that the current Quake servers have a similar frame cycle. They had to, to provide -listen server support. Even when you run a dedicated server, the model is still: fetch all input, process the world, send updates out to all clients. The default server framerate is 20 fps (50 ms). You can change this by adjusting the sys_ticrate cvar, but there are problems either way. If you ask for more fps from the server, you may get less latency, but you would almost certainly overcommit the bandwidth of a dialup link, resulting in all sorts of unwanted buffering in the routers and huge multi thousand ms latency times as things unclog (if they ever do).
The proper way to address this is by changing the server model from a game style loop to a fileserver/database style loop.
Instead of expecting everyone's messages to be dealt with at once, I now deal with each packet as it comes in. That player alone is moved forward in time, and a custom response is sent out in very short order. The rest of the objects in the world are spread out between the incoming packets. There are a lot of issues that that brings up. Time is no longer advancing uniformly for all objects in the world, which can cause a lot of problems.
It works, though! The average time from a packet ariving at the system to the time a response is sent back is down to under 4ms, as opposed to over 50 with the old dedicated servers.
Another side benefit is that the server never blindly sends packets out into the void, they must be specifically asked for (note that this is NOT a strict request/reply, because the client is streaming request without waiting for the replies).
I am going to be adding bandwidth estimation to help out modem links. If quake knows that a link is clogged up, it can choose not to send anything else, which is far, far better than letting the network buffer everything up or randomly drop packets. A dialup line can just say "never send more than 2000 bytes a second in datagrams", and while the update rate may drop in an overcommited situation, the latency will never pile up like it can with the current version of quake.
The biggest difference is the addition of client side movement simulation.
I am now allowing the client to guess at the results of the users movement until the authoritative response from the server comes through. This is a biiiig architectural change. The client now needs to know about solidity of objects, friction, gravity, etc. I am sad to see the elegent client-as-terminal setup go away, but I am practical above idealistic.
The server is still the final word, so the client is allways repredicting it's movement based off of the last known good message from the server.
There are still a lot of things I need to work out, but the basic results are as hoped for: even playing over a 200+ ms latency link, the player movement feels exactly like you are playing a single player game (under the right circumstances -- you can also get it to act rather weird at the moment).
The latency isn't gone, though. The client doesn't simulate other objects in the world, so you apear to run a lot closer to doors before they open, and most noticably, projectiles from your weapons seem to come out from where you were, instead of where you are, if you are strafing sideways while you shoot.
An interesting issue to watch when this gets out is that you won't be able to tell how long the latency to the server is based on your movement, but you will need to lead your opponents differently when shooting at them.
In a clean sheet of paper redesign, I would try to correct more of the discrepencies, but I think I am going to have a big enough improvement coming out of my current work to make a lot of people very happy.
-----------------------------------------
John Carmack's .plan for Aug 08, 1996
-----------------------------------------
Romero is now gone from id.
There will be no more grandiose statements about our future projects.
I can tell you what I am thinking, and what I am trying to acomplish, but all I promise is my best effort.
-----------------------------------------
John Carmack's .plan for Aug 10, 1996
-----------------------------------------
QuakeWorld structural addendum:
After hearing many arguments against the single master server, ranging from coherent and well reasoned to paranoid whining, I now agree that the single global master server isn't sufficient.
During the R&D phase, there will still be only the single server, but after all the kinks get worked out, I will allow a small number of sites to run private master servers. This will not be a general release, but only to properly licensed third parties. That will still allow me to collect my 100% coverage data, and it will prevent a single network/computer/software failure from preventing all QuakeWorld play.
QuakeWorld technical addendum:
I am reining in the client side prediction to a fairly minimal amount. It has too many negative effects in different circumstances. The problem of running away from or in front of your missiles was so bad that I considered simulating the missiles on the client side, but that is the second step on a slippery slope. If just the missiles were simulated, then a missile would fire through an enemy until the server informed you it exploded on them. Then you consider simulating interactions, but then you have to guess at other player inputs (which is hopeless)...
Lesson learned: Simulating 300 ms on the client side in a Quake style game is just out of the question. It probably works fine for flight sim or driving sims, but not in out twitch reaction games.
I am currently using client side simulation to smooth out the beat frequency interactions between server packet arrival and client frame times. In the shipping version of Quake, some latency was introduced on purpose to keep the displayed frame simulation time between the last two packets from the server so that the variability in arrival time could be smoothed out. In QuakeWorld, I am always starting with the most current packet, and using simulation to smooth out the variability. This <100ms of client side motion seems to be very manageable, and cuts out some real latency as well as doing the gueswork.
It looks like I am going to split the QuakeWorld client into multiple threads to reduce the avg 1/2 frame latency between input and packet sending. This is also a step towards building a multi-threaded Quake renderer, which will let multi-cpu NT machines render twice as fast. Lets hope the windows thread scheduler is decent...
I have been cutting down the message sizes a bit here and there as well. On a modem link, every couple bytes I save translates into a millisecond of latency saved. I cut an average of 17 bytes from the server to client and 8 from the client to server today.
-----------------------------------------
John Carmack's .plan for Aug 12, 1996
-----------------------------------------
Apogee's Prey team (and Duke's Levelord) leave 3drealms to work with Quake technology as Hipnotic Interactive.
:-)
-----------------------------------------
John Carmack's .plan for Aug 13, 1996
-----------------------------------------
I am considering increasing the default sv_friction value for QuakeWorld from 4 to 6 or 8.
It might take a little getting used to, but I think it gives more precise control for wide area network play.
If anyone wants to run some experiments with different friction levels on a current Quake server, I would be interested in hearing some feedback.
-----------------------------------------
John Carmack's .plan for Aug 17, 1996
-----------------------------------------
The remote server console commands are fully implemented for QuakeWorld.
To allow remote administration, the server must set the "password" cvar. By default, remote administration is turned off.
On a client, if you set the "password" cvar to the same value, you can issue "rcon" commands to the remote server :
rcon <server command> <arguments> ...
You can go to different levels, shut the server down, change cvars, ban people, etc. The output from the command is redirected over the net and will be echoed on the remote console.
You can also execute commands without even connecting to the server (if it was full) by setting the "rconadr" cvar to the full internet address (including port) of the system you want to administer.
2:00 in the morning:
My testarossa snapped another input shaft (the third time). damn dman damn. >1000 HP is bad for your drivetrain.
-----------------------------------------
John Carmack's .plan for Aug 18, 1996
-----------------------------------------
PACKET FILTERING
QuakeWorld supports two types of filtering: IP packet filtering and user id filtering. Userid filtering is the most convenient way to keep a specific person off of a server, but because anyone can create as many accounts as they want, a malicious user could just keep logging back in with a new account. If their ip address is banned, they won't be able to log on with any account from that computer. Unfortunately, most dialup accounts give a different ip address for each connection, so you may be forced to ban an entire subnet to keep a specific person off.
You can add or remove addresses from the filter list with:
addip <ip> removeip <ip>
The ip address is specified in dot format, and any unspecified digits will match any value, so you can specify an entire class C network with "addip 192.246.40".
Removeip will only remove an address specified exactly the same way. You cannot addip a subnet, then removeip a single host.
iplist Prints the current list of filters.
writeip Dumps "addip <ip>" commands to iplist.cfg so it can be execed at a later date. The filter lists are not saved and restored by default, because I beleive it would cause too much confusion.
filterban <0 or 1>
If 1 (the default), then ip addresses matching the current list will be prohibited from entering the game. This is the default setting.
If 0, then only addresses matching the list will be allowed. This lets you easily set up a private game, or a game that only allows players from your local network.
-----------------------------------------
John Carmack's .plan for Aug 22, 1996
-----------------------------------------
The rendition 3d accelerated version of Quake looks very good.
The image quality is significantly better than software - dithered, bilinear interpolated textures, and subpixel, subtexel polygon models.
It is faster than software even at 320*200, and at 512*384 it is almost twice as fast.
We do take a bit of a hit when we have to generate a lot of 16 bit surfaces, so occasionally the framerate gets a bit uneven, but overall it is a very solid improvement.
-----------------------------------------
John Carmack's .plan for Aug 27, 1996
-----------------------------------------
I haven't been working on QuakeWorld for the past week, because Michael has been working on the rendition port, which backburnered the win32 native port, which is a prerequisite for the QuakeWorld release.
Instead, I have been working on some better graphics code.
I now have a radiosity replacement for ligting quake maps. This will have a good effect on our Quake 2 maps, but probably won't be feasable for homebrew use.
I am finishing up a shadow volume pre-subdivision stage right now, and I need to do the resampling to quake light maps a bit better than I currently am, but I am very pleased with how it has turned out. A lot of the work will play into the next generation (NOT quake 2) game technology, for things like colored radiosity and dynamic shadows, but it is still cool just for calculating quake light maps.
Instead of placing lots of floating light entities, certain textures are tagged as being light emitters, so if you draw a flourecent light in the ceiling, it will automatically emit light from it's entire area. Lava emits light from the entire surface.
The simulation of the light is realistic now, instead of just hacked together. Light reflects off of surfaces, so ceilings will get lit even if there is no light pointing directly at them. The way light flows around corners looks eerily realistic.
I fixed the sky volumes so missiles dissapear inside them again, which was necessary to implement a new lighting feature: you can specify the position of the sun (or moon) in degrees, and the amount of light it will emit. The light then comes out completely parallel and at a constant intensity, passing through sky volumes to light up all outdoor areas in a consistant way. (this could be retrofited onto light.exe for people unable to run qrad)
Now for the bad news: a full size level takes several minutes to process on our quad alpha, and you MUST do a vis before running qrad (otherwise it would take about 1000 times longer). This wouldn't be out of reach even on mortal pentium systems (you can ask for a rough aproximation instead of the full job), except for the working set size. >100 megs. I'm NOT being wasteful here, either. I use a halved bit array for visibility interconnects, and sparse scaled shorts for energy transference.
I will release the source after we have been using it in production for a while, in case anyone wants to take a shot at reformulating it to be less memory intensive. It would be possible to get it down to 32 megs with some reduction in quality and a lot more wasted time, but then the run time would be obnoxious. I expect homebrew levels are going to have to stick with light.exe.
I am looking forward to doing a little more work on the other utilities as well. I want to break qbsp up into three seperate programs, so it will be able to run on machines with <32 megs of memory. Light can be sped up significantly if the PVS information is taken advantage of the way I did in qrad. I also have some theories on changing the algorithmic order of full vis processing time that I want to look into, especially now that the vis stage is put before production lighting.
-----------------------------------------
John Carmack's .plan for Sep 03, 1996
-----------------------------------------
I have been cleaning up the utilities for another release (soon) where everything is in Visual C++ project format. All of the common code is now included from the same place, so adding support for pcx or 3d studio files will only require changes in one file now for all utilities.
I made a few simple changes to qcc, and it now compiles FOUR TIMES FASTER! A recompile on my pp200 only takes 5 seconds now. I knew the linear search for defs was hurting, but if I had realized it was that dominant (and easy to fix) I would have fixed it a long time ago. All I had to do was move defs to the head of the search list each time they were looked for, so they would be found fast the next time. A big hash table might be somewhat faster still, but this was trivial to add.
Qbsp now uses floats instead of doubles by default, which makes it a good deal less of a memory pig, but it seems to give numerical accuracy problems in some cases.
I am splitting qbsp up into three seperate programs, which will make the memory requirements much smaller than they are now, and will help me be more numerically rigorous. This is going to take some time, so it won't be in the next code release.
-----------------------------------------
John Carmack's .plan for Sep 10, 1996
-----------------------------------------
Ok, I'm back on QuakeWorld.
While working on the new chat console, I fixed up a few little things with the regular console:
The pgup/pgdn keys now autorepeat.
The display does not pop down to the bottom when a new line is printed.
If you aren't at the bottom, a few ^ ^ ^ are printed on the last line.
There is now a message level you can set to cut down on the number of messages you get from the server (it saves a bit of bandwidth, too).
Level 0 gives everything.
Level 1 does not print item pickup messages.
Level 2 does not print player obituaries.
Level 3 does not print any messages from the server.
Level 4 does not print chat messages.
Player skin support is much nicer in QuakeWorld. There is a qw\skins directory, where you can just put a .pcx files for each skin. This will scale properly to hundreds of skins if you want to download them all.
I am still unsure of how much freedom I want to give with the skins.
One school of thought is that you just set the "skin" value to any name you want. If the other players don't have it, they fall back to the default.
The other school of thought is that skins are the restricted property of a specific clan, and only clan members get custom skins.
We have the ability to delegate specific information setting abilities to clan leaders. To join a clan, you have to get the leader to add the clan field to your user account. It is in the protected masterdata, so you can't edit it yourself.
-----------------------------------------
John Carmack's .plan for Sep 13, 1996
-----------------------------------------
There is a BETA distribution of 1.05 up on our ftp site.
ftp://ftp.idsoftware.com/idstuff/unsup/q105beta.exe
It includes quake.exe winded.exe progs.dat, and a zip of the latest .qc files. Back up your current version before messing with this, in case something is broken.
There is new source code available. ftp://ftp.idsoftware.com//idstuff/source/qutils.zip has win-32 compiled versions and VC++ projects for all of the quake command line utilities. The qbsp/light/vis programs have not been tested all that well here, because we still use a dec alpha running unix for most of our work, but it has been converted from doubles to floats, which should reduce the memory requirements quite a bit (and possibly cause more numeric instabilities...).
I also uploaded the DOOM ipxsetup / sersetup source code, which we haven't had on our ftp site before.
We are going to begin internal testing of a 1.05 Quake release tomorrow. When we upload for general release, we will also upload the current .qc source. The old qcc.tar.gz file is still present on the ftp site for the original .qc files for now.
-----------------------------------------
John Carmack's .plan for Sep 18, 1996
-----------------------------------------
We are now beta testing the first release of QuakeWorld.
It seems to be going well. The latency reductions are very real, and significantly improve control and responsiveness. There is some loss of smoothness in some cases, but you can tune the tradeoff to your liking.
I don't want everyone to think that it will instantly turn their 28.8 modem into a lan connection, but it should be solidly better than what we have in all circumstances.
On an internet connection that provides a stable flow of packets at a reasonably predictable rate, the gameplay is very good. Single packet drops are not even noticable, but dropping several packets at once or having a large variability in latency can still mess up a game.
The really cool ISPs will run QuakeWorld servers right at their pops, which should give 28.8 modem players 200 ms or less pings, providing a nearly flawless game.
-----------------------------------------
John Carmack's .plan for Sep 22, 1996
-----------------------------------------
I had a good time at the New York event -- it was fun to meet a lot of the web and clan guys, and there was some good fierce deathmatch action.
The QuakeWorld beta has revved a couple more times. When things work right, its great, but it seems like it only works right on about half the windows machines we try it on.
We have at least a week of beating on it before a public release.
-----------------------------------------
John Carmack's .plan for Oct 14, 1996
-----------------------------------------
I haven't been able to work on QuakeWorld for a week, and am unlikely to for a couple more days.
I've been working with the utilities and editor most of the time, but there has been a lot of other stuff going on lately that has kept me from being very productive.
-----------------------------------------
John Carmack's .plan for Oct 16, 1996
-----------------------------------------
The latest windows drivers from rendition fix the lousy GUI performance I had been complaining about to several people. While it may not be state-of-the-art 2d acceleration, it doesn't get in your way anymore. I do not consider windows performance to be a negative for rendition anymore.
-----------------------------------------
John Carmack's .plan for Nov 06, 1996
-----------------------------------------
Over the weekend, I ported Quake to open-gl / win32. It is a complete port with all features supported. Its not really practical right now on most consumer level equipment, but it runs ok on my intergraph glz-13t. It will work on the 3dlabs cards with their next driver release, but it really isn't very optimized yet, so don't expect anything really fast.
If you have high end hardware (intergraph realizm or dec powerstorm), it is pretty cool.
It is interesting to look at quake with hires 24 bit trilinear texture mapping. Sometimes you just don't even notice anything is different, but sometimes you can just stare at a scene for quite a while apreciating it.
Throwing bits and pixels at a static design certainly doesn't hurt, but it really doesn't bring it to a new level. I think I know where the next level is, but I'm just at the very beginning of the work on the next game architecture.
We have three mostly-done things to release: winquake, glquake, and quakeworld. No, I don't know when any of them are actually going to be released. Sorry.
QuakeWorld will actually download a new level for you now if you want to wait for it. Little models and sounds are a lot more practical to customize a server with, though.
Today I got cornered by the three level designers and an artist and they jumped all over me for working on new research instead of polishing the quake 2 tools. I guess the rest of my week is spoken for...
-----------------------------------------
John Carmack's .plan for Nov 23, 1996
-----------------------------------------
QuakeWorld: there is one physics problem that is preventing me from releasing: you get stuck sometimes in moving plats. I am having a hard time tracking it down, and I only get to work on it sporadically. I will spend a bit more time this weekend.
I have given the source for the master server to the QSpy guys and Disruptor to do with as they will (rankings, queries, etc). I know I won't have the time to support it like it deserves.
WinQuake: Michael is almost done.... We only have one reported serious problem, but there are still a list of little issues. We are finally feeling stable on windows. Not everyone may be able to get optimal settings, but SOMETHING should work almost everywhere.
I have an NT alpha machine on order, so we will be compiling for that as well. We aren't going to be able to spend time writing assembly language for it like the intel version has, but it should still run at a pretty decent clip. IBM was talking about getting us a power-PC machine, but I haven't heard back from them in a while. If one shows up, we will compile for that as well.
GLQuake: 3DFX is writing a mini-gl driver that will run glquake. We exepect very high performance. 3Dlabs is also working with me on improving some aspects of their open-GL performance. DynamicPictures sent me an Oxygen board, but glquake did a horrible crash and burn with their beta drivers. They are looking into it. DEC is sending me a powerstorm to work with, which should perform in the same ballpark as the intergraph realizm I did the development work on.
I do expect the 3dfx version to outperform the $5k+ professional gl cards, due to its higher fill rate and the fact that they can tune directly for quake. There are certainly valid reasons to buy $5k cards (24+ bit z buffers, 24 bit color, 1280 res, etc), but don't cry when a $300 card eats their lunch :-).
Because of the very fill-rate intensive nature of the way I implemented this version of glquake (using a blend pass to do the light maps), performance is not likely to be very good on any current generation consumer level board except 3dfx. The next generation of most vendors cards should have sufficient fill rate, if claims are to be believed. I may do a surface cached version just for the experience, in any case.
One little public rant: if you are a 3d chip designer, don't even THINK about not supporting true alpha blending. Screen door transparency is not a valid replacement. (this means YOU, matrox!)
IRIX-GLQuake: In two weeks, Ed Hutchins from SGI is coming down and we are going to port glQuake to irix. This will only run (reasonably) on systems with hardware texture mapping: O2, impact, RE2, and IR. No indys, extremes, etc. You could probably use them as dedicated servers, though.
D3DQuake: I started bringing up glquake on direct-3d a couple days ago. We are going to have a very nice, apples-to-apples api comparison here. It takes four times as much code to do anything in D3D as it does in OpenGL, but I will tune both to the best of my abilities and we shall see if there is any performance advantage to either API. We should be able to do comparisons on both 3DFX and 3DLabs hardware with both apis.
Quake utilities: I have split qbsp into two programs: qcsg and qbsp2. The processing is more robust, faster, and more memory frugal, but it currently generates rathar unoptimized maps. I will be writing a seperate map optimizer program soon to get the counts and size back to what they should be. I will probably do a new tool release after we get some more testing time on them. All of the utilities except qbsp now automatically use as many processors as you have if you are running on NT.
New stuff: The outline of the next generation game engine is beginning to take a bit of shape... I'm applying the lessons learned from quake and I have a long list of new things to try. I want to polish off all the outstanding quake stuff, then just go into hermit mode and work on the future while the other guys do Quake 2.
-----------------------------------------
John Carmack's .plan for Dec 13, 1996
-----------------------------------------
I need to clear up a little bit of really flagrant misinformation here:
Digital Sight and Sound, a dallas based system integrator, run a little blurb in their customer update newsletter that seriously misrepresents my experiences with intergraph and sgi hardware.
The title was "Z-13 versus sgi maximum impact, Z-13 wins" and it included statements like "After two weeks of grueling tests, the z-13 finished 2.5 times faster overall than the Maximum IMPCT... and at less than 1/2 the price!!!". This just isn't true, in saveral ways.
The last time I did side by side system testing involving SGI was pitting an SGI high impact against a intergraph GLZ1T. For heavy texture mapping applications like I was doing, the SGI high impact was three times faster than the intergraph. A realizm-13t is 2.5 times faster than the older GLZ1T, but a max impact is twice as fast as a high impact, so intergraph's best is still only half the speed of sgi's best workstation.
For the record, here is my impression of the 3D hardware I have worked with, unedited. Prices are for configurations as we would use them, not minimum.
Rendition Verite: ($150) It won't set any speed records, but it is an excellent value, the quake port is pretty good, and the risc architecture allows a lot of room for optimization.
3DFX: ($300) The highest textured fill rate of anything short of high end SGI hardware. 50 mpix, almost irrespective of options. You can't run a desktop or high res display on it, but at 640*480 it will stomp any other consumer level board, given equal optimization effort.
3dlabs permedia: ($300) Right now, this is the only well supported low end open GL card. and a good all around video card. The visual quality is hurt over most other 3d boards because the internal paths apear to be 15 bits, so you don't get dithered 24 bit color out of your blending operations. The fill rate is very sensitive to what options (z buffer, blending, color depth, etc) you have turned on. It peaks at over 40 mpix, but that is with no zbuffering and 4 bit textures. Realistic cases (16 bit textures, z buffering, perspective, bilinear) bring you down to around 10 mpix. There are some performance hangups in the drivers for heavy texturing applications, but 3dlabs is working on improving them.
The remaining boards are pretty much for professional use, not gaming.
Dynamic Pictures Oxygen: ($1500) I can't reccomend this to anyone doing texture mapping. It was slow and had rasterization errors.
3dlabs glint-TX: ($1000 - $3000) A reletively low fill rate (15 mpix peak, often a bit lower), but fairly stable in performance. Good GL drivers. Capable of supporting very high resolutions and amounts of texture memory. Available from several vendors at various (professional) price points.
Intergraph intense-3d: ($2000 - $5000). The fill rate still isn't very good (15 mpix), but it hardly cares when you turn on trilinear and blending. They rape you on the texture memory prices. I have gotten better performance (but not tons better) out of this than any glint card I have tested, but I have yet to use one of the dual TX boards. They are also incompatable with a lot of motherboards.
Intergraph realizm: ($7000 - $12000) The best graphics card you can buy for NT. As far as I know, you can't even get these seperately, you have to buy them as part of an intergraph system. The systems we use list around $30k each. The fill rate is 33 mpix, and very insensitive to options. You can add an optional geometry board to get very high transform/light throughput. DEC resells these as their Powerstorm T series cards for alpha systems.
SGI O2: ($10000, full system) The fill rate was better than intergraph's high end, but the triangle throughput was lower. I expect SGI will be able to optimize the porformance more in the coming months. It's not a knock-your-socks-off performer, but a real, fully configured system costs less than just the video option on a high end intergraph.
SGI IMPACT: ($25000 - $50000, full system) excellent fill rate and geometry rate. A very balanced, high performace system. The texture memories are very small (one or four megs), so it would probably be possible to contruct a pathalogical case that probably puts an intergraph ahead of it, but you would have to stretch. The prices are higher than intergraph, but only by about 30%.
SGI Infinite reality: ($100000+) Fill rate from hell. Polygons from hell. If you don't trip up on state changes, nothing will come within shouting distance of this system. You would expect that.
Our decision to go intergraph/NT over SGI/IRIX had a lot more to do with the NT vs IRIX part of it. I wish I could buy SGI hardware for NT. The last two SGI systems I did serious work on were messes of instability, but the O2 we just got last week does actually seem stable.
As far as the "which video card should I buy?" question from gamers, I'm afraid I can't give you a definitive answer. All of the first generation boards have some problem or another. The second generation, coming out within the next six months or so, will be more solidly positive.
-----------------------------------------
John Carmack's .plan for Dec 17, 1996
-----------------------------------------
QuakeWorld went up, got to around 4000 users, then the master server exploded.
Disrupter and cohorts are working on more robust code now.
If anyone did it on purpose, how about letting us know... (It wasn't all the people that tried %s as a name)
-----------------------------------------
John Carmack's .plan for Dec 21, 1996
-----------------------------------------
Wow. Apple bought NeXT. That really brightened my day.
I haven't generally been unhappy developing on NT, but I had been carrying a bit of sadness over several of the things we left behind when we moved from NEXTSTEP.
I wouldn't touch a mac for development right now, but if apple does The Right Thing with NeXT...
-----------------------------------------
John Carmack's .plan for Dec 23, 1996
-----------------------------------------
OpenGL vs Direct-3D
I am going to use this installment of my .plan file to get up on a soapbox about an important issue to me: 3D API. I get asked for my opinions about this often enough that it is time I just made a public statement. So here it is, my current position as of december '96...
While the rest of Id works on Quake 2, most of my effort is now focused on developing the next generation of game technology. This new generation of technology will be used by Id and other companies all the way through the year 2000, so there are some very important long term decisions to be made.
There are two viable contenders for low level 3D programming on win32: Direct-3D Immediate Mode, the new, designed for games API, and OpenGL, the workstation graphics API originally developed by SGI. They are both supported by microsoft, but D3D has been evangelized as the one true solution for games.
I have been using OpenGL for about six months now, and I have been very impressed by the design of the API, and especially it's ease of use. A month ago, I ported quake to OpenGL. It was an extremely pleasant experience. It didn't take long, the code was clean and simple, and it gave me a great testbed to rapidly try out new research ideas.
I started porting glquake to Direct-3D IM with the intent of learning the api and doing a fair comparison.
Well, I have learned enough about it. I'm not going to finish the port. I have better things to do with my time.
I am hoping that the vendors shipping second generation cards in the coming year can be convinced to support OpenGL. If this doesn't happen early on and there are capable cards that glquake does not run on, then I apologize, but I am taking a little stand in my little corner of the world with the hope of having some small influence on things that are going to effect us for many years to come.
Direct-3D IM is a horribly broken API. It inflicts great pain and suffering on the programmers using it, without returning any significant advantages. I don't think there is ANY market segment that D3D is apropriate for, OpenGL seems to work just fine for everything from quake to softimage. There is no good technical reason for the existance of D3D.
I'm sure D3D will suck less with each forthcoming version, but this is an oportunity to just bypass dragging the entire development community through the messy evolution of an ill-birthed API.
Best case: Microsoft integrates OpenGL with direct-x (probably calling it Direct-GL or something), ports D3D retained mode on top of GL, and tells everyone to forget they every heard of D3D immediate mode. Programmers have one good api, vendors have one driver to write, and the world is a better place.
To elaborate a bit:
"OpenGL" is either OpenGL 1.1 or OpenGL 1.0 with the common extensions. Raw OpenGL 1.0 has several holes in functionality.
"D3D" is Direct-3D Immediate Mode. D3D retained mode is a seperate issue. Retained mode has very valid reasons for existance. It is a good thing to have an api that lets you just load in model files and fly around without sweating the polygon details. Retained mode is going to be used by at least ten times as many programmers as immediate mode. On the other hand, the world class applications that really step to new levels are going to be done in an immediate mode graphics API. D3D-RM doesn't even really have to be tied to D3D-IM. It could be implemented to emit OpenGL code instead.
I don't particularly care about the software only implementations of either D3D or OpenGL. I haven't done serious research here, but I think D3D has a real edge, because it was originally designed for software rendering and much optimization effort has been focused there. COSMO GL is attempting to compete there, but I feel the effort is misguided. Software rasterizers will still exist to support the lowest common denominator, but soon all game development will be targeted at hardware rasterization, so that's where effort should be focused.
The primary importance of a 3D API to game developers is as an interface to the wide variety of 3D hardware that is emerging. If there was one compatable line of hardware that did what we wanted and covered 90+ percent of the target market, I wouldn't even want a 3D API for production use, I would be writing straight to the metal, just like I allways have with pure software schemes. I would still want a 3D API for research and tool development, but it wouldn't matter if it wasn't a mainstream solution.
Because I am expecting the 3D accelerator market to be fairly fragmented for the forseeable future, I need an API to write to, with individual drivers for each brand of hardware. OpenGL has been maturing in the workstation market for many years now, allways with a hardware focus. We have exisiting proof that it scales just great from a $300 permedia card all the way to a $250,000 loaded infinite reality system.
All of the game oriented PC 3D hardware basically came into existance in the last year. Because of the frantic nature of the PC world, we may be getting stuck with a first guess API and driver model which isn't all that good.
The things that matter with an API are: functionality, performance, driver coverage, and ease of use.
Both APIs cover the important functionality. There shouldn't be any real argument about that. GL supports some additional esoteric features that I am unlikely to use (or are unlikely to be supported by hardware -- same effect). D3D actually has a couple nice features that I would like to see moved to GL (specular blend at each vertex, color key transparancy, and no clipping hints), which brings up the extensions issue. GL can be extended by the driver, but because D3D imposes a layer between the driver and the API, microsoft is the only one that can extend D3D.
My conclusion about performance is that there is not going to be any significant performance difference (< 10%) between properly written OpenGL and D3D drivers for several years at least. There are some arguments that gl will scale better to very high end hardware because it doesn't need to build any intermediate structures, but you could use tiny sub cache sized execute buffers in d3d and acheive reasonably similar results (or build complex hardware just to suit D3D -- ack!). There are also arguments from the other side that the vertex pools in d3d will save work on geometry bound applications, but you can do the same thing with vertex arrays in GL.
Currently, there are more drivers avaialble for D3D than OpenGL on the consumer level boards. I hope we can change this. A serious problem is that there are no D3D conformance tests, and the documentation is very poor, so the existing drivers aren't exactly uniform in their functionality. OpenGL has an established set of conformance tests, so there is no argument about exactly how things are supposed to work. OpenGL offers two levels of drivers that can be written: mini client drivers and installable client drivers. A MCD is a simple, robust exporting of hardware rasterization capabilities. An ICD is basically a full replacement for the API that lets hardware accelerate or extend any piece of GL without any overhead.
The overriding reason why GL is so much better than D3D has to do with ease of use. GL is easy to use and fun to experiment with. D3D is not (ahem). You can make sample GL programs with a single page of code. I think D3D has managed to make the worst possible interface choice at every oportunity. COM. Expandable structs passed to functions. Execute buffers. Some of these choices were made so that the API would be able to gracefully expand in the future, but who cares about having an API that can grow if you have forced it to be painful to use now and forever after? Many things that are a single line of GL code require half a page of D3D code to allocate a structure, set a size, fill something in, call a COM routine, then extract the result.
Ease of use is damn important. If you can program something in half the time, you can ship earlier or explore more aproaches. A clean, readable coding interface also makes it easier to find / prevent bugs.
GL's interface is procedural: You perform operations by calling gl functions to pass vertex data and specify primitives.
[code]
glBegin (GL_TRIANGLES);
glVertex (0,0,0);
glVertex (1,1,0);
glVertex (2,0,0);
glEnd ();
[/code]
D3D's interface is by execute buffers: You build a structure containing vertex data and commands, and pass the entire thing with a single call. On the surface, this apears to be an efficiency improvement for D3D, because it gets rid of a lot of procedure call overhead. In reality, it is a gigantic pain-in-the-ass.
[code]
(psuedo code, and incomplete)
v = &buffer.vertexes[0];
v->x = 0; v->y = 0; v->z = 0;
v++;
v->x = 1; v->y = 1; v->z = 0;
v++;
v->x = 2; v->y = 0; v->z = 0;
c = &buffer.commands;
c->operation = DRAW_TRIANGLE;
c->vertexes[0] = 0;
c->vertexes[1] = 1;
c->vertexes[2] = 2;
IssueExecuteBuffer (buffer);
[/code]
If I included the complete code to actually lock, build, and issue an execute buffer here, you would think I was choosing some pathologically slanted case to make D3D look bad.
You wouldn't actually make an execute buffer with a single triangle in it, or your performance would be dreadfull. The idea is to build up a large batch of commands so that you pass lots of work to D3D with a single procedure call.
A problem with that is that the optimal definition of "large" and "lots" varies depending on what hardware you are using, but instead of leaving that up to the driver, the application programmer has to know what is best for every hardware situation.
You can cover some of the messy work with macros, but that brings its own set of problems. The only way I can see to make D3D generally usable is to create your own procedural interface that buffers commands up into one or more execute buffers and flushes when needed. But why bother, when there is this other nifty procedural API allready there...
With OpenGL, you can get something working with simple, straightforward code, then if it is warranted, you can convert to display lists or vertex arrays for max performance (although the difference usually isn't that large). This is the right way of doing things -- like converting your crucial functions to assembly language after doing all your development in C.
With D3D, you have to do everything the painful way from the beginning. Like writing a complete program in assembly language, taking many times longer, missing chances for algorithmic improvements, etc. And then finding out it doesn't even go faster.
I am going to be programming with a 3D API every day for many years to come. I want something that helps me, rather than gets in my way.
John Carmack
Id Software
-----------------------------------------
John Carmack's .plan for Jan 12, 1997
-----------------------------------------
We now have someone officially in charge of quake/quakeworld unix ports:
Dave 'Zoid' Kirsch. (zoid@threewave.com)
Direct any correspondance about the linux ports of quake to him. If any other unix vendors would like to have quake ported to their environments, set up an equipment loan with him. This is a volenteer position, so don't give him a hard time.
-----------------------------------------
John Carmack's .plan for Jan 22, 1997
-----------------------------------------
A preliminary release of glquake and the 3dfx driver has been put on our ftp site in the idstuff/unsup directory.
one hour later...
3dfx gave me a new vxd for glquake that fixes crashing problems on some pentium pro systems. glquake1.zip now contains the current files.
-----------------------------------------
John Carmack's .plan for Jan 25, 1997
-----------------------------------------
I fixed a couple problems in glquake with some addon stuff -- the goats demo actually showed up a crashing problem with 256*256 textures, and the rangers demos showed that players with non-player models and colormaps were all messed up.
I'll make another release when 3dfx has their next driver drop finished.
-----------------------------------------
John Carmack's .plan for Jan 31, 1997
-----------------------------------------
I went down to the ferrari dealership today with Ann and American with the intention of buying a new f355 as a "sensible" car (in contrast to my other twin turbo monstrosities).
There was an F40 sitting out front.
Umm. Sensible car. F40. Sensible car. F40. Hmmmm.
American played the evil conscience and Ann played the good conscience. For a while. Then Ann got infected by the Dark Side. :-) I bought the F40.
Ok, I now have too many ferraris. I have a plan to fix that. This hasn't gone through all the legal crap necessary yet, but it is my intention to give away my first ferrari as the grand prize of a Quake tournement.
Yes, I am serious.
DO NOT SEND ME ANY MAIL ABOUT THIS! When we have more to say about it, we will make a formal announcement. This would be a good time to start a heated debate in the newsgroups about what the fairest tourney rules would be, though.
The specs:
1987 328 GTS with turbocharged engine.
Feature in the january, 1994 issue of Turbo magazine.
Made 360 HP at the rear wheels on a chassis dyno.
New engine (I melted the first one...), new paint.
The interior could use a little work.
I plan to also include enough cash to cover tax and insurance.
-----------------------------------------
John Carmack's .plan for Feb 05, 1997
-----------------------------------------
I've been following the ferrari discussion on r.g.c.q, and here are a couple clarifications:
The final rounds will definately be one on one deathmatch over a LAN, probably double elimination.
The big question is how do we get the number of people who would like to take a shot at it down to a reasonable (64?) number of people for a proper tournement.
We don't know yet where the final rounds are going to be held, either. E3? #Quakecon? A microsoft event? A totally specific event?
I know that whatever we come up with won't be 100% fair to our entire customer base, but it would be really unfortunate if some people got bitter over it.
-----------------------------------------
John Carmack's .plan for Feb 13, 1997
-----------------------------------------
There will be a new QuakeWorld release soon with a very different interface and several improvements.
On the topic of network gaming, I have a couple questions that I would like answered by anyone with the required technical knowledge:
If I understand correctly, UDP headers are currently not compressed over PPP links like TCP headers commonly are. During a network game (or internet phone or videoconference), all the packets are just between two addresses, but tons of bandwidth (and corresponding latency) is eaten by sending full headers. Has anyone developed UDP header compression extensions, and if so, how common are they?
On a lower hardware level, have any modem manufacturers considered optimizing interfaces or protocols for lower packet latency? Once again, if I understand correctly, modem transmission involves a level of packetizing for modulation/protocol/compression work that is underneath the serial interface seen by a computer. Do partially filled buffers sit idle for a time if there isn't a continuous stream of data, or do they immediately flush when a byte failes to clock in? It would be nice to be able to explicitly syncronize with UDP packets, because a realtime game running within bandwidth limits is going to send and receive discrete packets interspersed with idle time, which is not really what modems are optimized for now. I know that modem data compression is generally a bad thing for low latency connections right now, but if the modem could be signaled to flush at UDP boundaries, it would probably become a positive thing. I'm sure there are also things that could be done at the modulation/protocol layer to improve latency at some cost in bandwidth or error rate, but I have no idea if it could be done compatably.
It would be an interesting point of differentiation for ISP's or modem manufacturers if an "optimize for games" checkbox was available somewhere. Even at the os level, a "max packets buffered" option would be valuable to keep packets from piling up when bandwidth is overcommited.
Comments?
-----------------------------------------
John Carmack's .plan for Feb 18, 1997
-----------------------------------------
Ferrari update.
Unless someone can come up with a compelling objection, I think we have an end plan now:
Intergraph is currently holding a very cool clan tournement that will have the finals at E3. The ClanRing guys are managing the tournement, and intergraph is picking up the bill. It's going to be great, with clan banners and projection screens. They allready have a pretty good prize lineup of cash and systems.
The current proposal is to expand the event to include one on one deathmatches in addition to the clan fights. The grand prize will be my 328. I don't think it would go over too well to have a ferrari as clan property :-)
Intergraph is willing to pay for travel and accomadations for twelve to sixteen contestants.
We still need to figure out how to select the finalists. The idea is that we can tell all the press types about the event now, and have a last-minute runnoff of some sort the month before E3. That way even people who aren't usually active on the internet can find out about it through a magazine and take a shot at it.
-----------------------------------------
John Carmack's .plan for Feb 19, 1997
-----------------------------------------
I made significant improvements to the scalability of QuakeWorld.
You have probably noticed how a 16 player game is a lot worse over a modem than a 4 player game, even when you aren't around the action. All of those factors are now gone, so big games play a lot better now, and I have bumped the maximum players to 32. There aren't really any levels that will be reasonable with that many players, but if someone wants to create one, it should be possible now. (obviously you are still going to go slow if all 32 people decide to congregate in the same room)
The next release of QuakeWorld is going to be completely incompatable with the current release. Both the client and server have changed protocols, and the fate of the master server is still undecided.
I have one more really significant thing to try with the network protocols that I should probably hold up the release for, because it would be yet another incompatable change.
-----------------------------------------
John Carmack's .plan for Feb 23, 1997
-----------------------------------------
I took an entire day and a half off from work to spend some quality time with my F40 at Texas World Speedway. I had a lovely moment pitching my quarter million dollar car off the track backwards (no harm done), but otherwise I had a great time. Back to work now.
I should have updates of both QuakeWorld and GlQuake within a week. QW is crapping out at 27 players for some reason (rather difficult to debug...), and glquake needs a fix for the hipnotic level pack to work.
My next project is to define a new rendering architecture that will clean a bunch of stuff up and allow me to combine regular quake, glquake, and a windows version of vquake into a single executable. I plan on doing the development work with QW, so I won't be stepping on Michael or Cash's toes as I go hacking and slashing through the codebase. If everything goes well, that will become the new guts of Quake 2, and I will probably also release a unified Quake 1 executable.
-----------------------------------------
John Carmack's .plan for Mar 05, 1997
-----------------------------------------
I have been soooo busy lately. (yes, thats my excuse for not having new glquake and quakeworld release yet)
Glquake works on the hipnotic pak now, but I still need to make a couple more changes. 3dfx has a new opengl.dll nearing completion.
QuakeWorld should be released soon, synced with a new version of qspy and CTF progs. I thought I was in feature freeze a week ago, but day before yesterday I couldn't help myself and I started implementing the last big thing I know how to do to improve the net play. I might as well get all the incompatible stuff done in one release, rather than breaking things again later. This release should make a lot of people happy.
We got our new SGI Origin-2000 server, and I have been tuning all the tools to work better on eight processors. I really wanted to buy an intel system running NT, but all of the big (8-16 processor) systems from Sequent, Data General, and NCR have incredibly brain damaged price tags assigned to them (in the neighborhood of $40,000 PER PROCESSOR).
I finally broke down and wrote a 3D model painting program for our artists. We have tried four seperate commercial programs, and none of them really do what we want. If you want something done right... The models in Quake2 are looking better in a lot of ways -- more polygons, more texels, more exacting texture work, smoother movement, etc.
I have been doing some cleanup work on our map editor, because we may wind up bundling it with an OpenGL accelerator card in the reletively near future. The terms aren't worked out yet, so I can't say much more.
Aaron Seeler from midway is coming down in a few days and we are going to start thrashing out the architecture for Quake on the N64. I'm really looking forward to that -- I have some clever things in mind that should really leverage the N64 hardware and deliver an awesome game. All of our previous cart ports have been just trying (and failing sometimes...) to equal a mid range PC game experience, but Quake on the N64 should be a lot closer to the vquake / glquake versions than the vanilla software version that most people are fammiliar with.
-----------------------------------------
John Carmack's .plan for Mar 11, 1997
-----------------------------------------
The new glquake is on ftp.idsoftware.com. It should work for both the hipnotic pack and the upcoming rogue pack.
/idstuff/unsup/glq3_11.zip
-----------------------------------------
John Carmack's .plan for Mar 12, 1997
-----------------------------------------
Ok, this really pisses me off:
[code]Date: Wed, 12 Mar 1997 13:32:43 +0100
From: "%Peter Lawrenz" <%Lawrenz@bln.de>
Reply-To: %Lawrenz@bln.de
X-Mailer: Mozilla 3.01 (Win95; I)
To: johnc@idsoftware.com
Subject: Re: E3/Ferrari Event PRESS RELEASE
Will Bryant wrote:
> John Carmack, the co-founder of id Software and creator of
> the game QUAKE, will give away one of his four Ferraris as
> the grand prize: a red 1987 Ferrari 328 GTS with a Norwood
> Autocraft turbo conversion. According to Carmack, donating
> the Ferrari is his way of giving something back to the game
> enthusiasts that have brought him success. "I bought my first
> Ferrari after the success of Wolfenstein-3D," said Carmack.
> "DOOM and QUAKE have bought three more. Four Ferraris is too
> many for me. Rather than sell off one of them or stick it in a
> warehouse, I'm going to give it back to the gamers that brought
> it to me in the first place. The king of this QUAKE deathmatch
> is going to get a really cool crown." "This is the biggest,
> hottest tournament ever for PC gamers, and we encourage all QUAKE
> players to give it their best shot," said Wade.
>
> Tournament Registration and Contact Information QUAKE players who
> want to participate in the tournament can register online at
> <mpog.com/e3signup.html>, anytime between March 24 and
> May 2, 1997. Participation is limited to US residents 18 years of
> age and older, other rules are posted on the Website.
Hi,
the two above statements contradict each other. How many ferraries
would you have been able to buy if the sales of your games were limited
to U.S. only ? It seems to me that you don't value your oversea
customers at all and invite them to pirate future ID software products.
I paid for Doom I, Doom II and Quake and since this press release I am
not really sure if I should spend any more money on ID software.
There are other companies that might value their customers for real.
-----
_|_ Sometimes you lose, sometimes the others win.
---*--O--*---
O O X Peter Lawrenz (DFV3772)
X X Quake + Spaceorb360 = FUN
X lawrenz@remove.the.%.from.the.reply-to
FreeFall-> FreeBag-> FreeBeer[/code]
AAAAARRRRGGGHHHH!!! I have gotten a few of these "you suck cuz I can't win your car" type of messages.
I'm giving the car away over the objections of my lawyer and my girlfriend just because I think it will be a cool and memorable thing to do. I want it to be as fair as possible, but I just don't have the time to spare to try and manage it myself. In any case, it wouldn't be fair to everyone no matter what we did. Sure, it would be great if we could fly the 10,000 interested people to a football stadium and have thousands of simultanious double ellimination bouts, but it just isn't going to happen.
I pushed against the "us citizans" clause, asking if it would be reasonable to bring a couple people from europe or austrailia, or even just allow some people that covered their own expenses, but aparently we would have to deal with the local laws in each country that an entrant comes from, which just wouldn't be reasonable for us. Even dealing with canada (as we have found out recently with Christian) can be a big pain in the ass.
Its just a fact of life that locality is an issue. We can't treat the entire world the same. Go convince another company that "might value their customers for real" to give you a ferrari.
-----------------------------------------
John Carmack's .plan for Mar 13, 1997
-----------------------------------------
Here is a technical issue to be discussed:
I am strongly considering dropping qc in Quake 2 in favor of exporting most of the game logic to a seperate .dll file. This wasn't an option when we had to support dos, but I think it is the correct choice now.
There are a lot of issues involved with this.
As everyone who has tried to do anything serious with qc knows, it has its limitations (ahem). I could improve the language, or just adopt a real language like java, but the simplest thing to do would be just use native code.
It would definately be more efficient as a dll. As we do more sophisticated game logic, efficiency becomes more and more important. For simple deathmatch modifications this wouldn't be a big deal, but for full size game levels it will likely be at least a 5% to 10% overall speed improvement.
It would be non-portable. I am dreading the reaction to this from the linux community. Game modifications would have to be compiled seperately for each target architecture (windows, linux, irix, etc). I do still intend to have the client be generic to all mods (but more flexible than Q1), so it is really only an issue for servers.
There are security concerns. I suppose to a world that embraces Active-X, this isn't really an issue, but binary code patches still spook me.
You would actually need a compiler to hack quake. For the serious people, this isn't an issue, but it would cut out a number of people that currently enjoy hacking quake. I have a strange mixture of pride and shame when I think about the people that have actually started learning programming in my crappy little qc language.
You could debug your patch in a real debugger! Yipee!
-----------------------------------------
John Carmack's .plan for Mar 18, 1997
-----------------------------------------
I have gotten a significant amount of response on the Quake 2 extension mechanism. I do read everything that comes my way (I can't respond to all of it, though), and I have learned a few things from the mail.
Nothing is set in stone yet, but it is still looking like a dll is going to be the primary interface. I have been seriously considering a java interface, but the tradeoffs (time spent implementing takes away from something else..) just don't quite add up. Other options, like enhancing qc or using other languages like perl have very remote chances.
One of the primary reasons is that you can allways build UP - put more functionality on top of a dll, but you can't allways build DOWN - accessing the registry from java for instance.
For Id Software to develop a game, a dll will be most efficient. We have more cpu power, and we can debug it more easily. We are directing significant effort towards making Quake 2 a better GAME, as well as just a better mutliplayer virtual world. Quake 1 was pretty messed up from a game standpoint, and we don't plan on doing that again.
What I can offer the qc hacking crowd is a public release of the qc interface and interpreter code from Quake 1 when Quake 2 is released. The user community can then bolt things together so that there can be one publicly trusted DLL that executes an updated and modified qc language for portable, secure add ons.
I really do care about portability, but it is just one factor that needs to be balanced against all the others. Things just aren't clear cut.
Speaking of portability, to remove the guesswork that goes on, here are my current opinions on the various platforms:
Win32
Win32 rules the world. You are sticking your head in the sand if you think otherwise. The upside is that windows really doesn't suck nowdays. Win 95 / NT 4.0 are pretty decent systems for what they are targeted at. I currently develop mostly on NT, and Quake 2 will almost certainly be delivered on win32 first. Our games should run as well as possible in NT, we won't require any '95 only features.
Dos
We are not going to do another dos game. No amount of flaming hate mail is going to change my mind on this (PLEASE don't!). The advantages of good TCP/IP support, dynamic linking, powerfull virtual memory, device drivers, etc, are just too much to overcome. Yes, all of those can be provided under dos in various ways, but it just isn't worth it.
Linux
I consider linux the second most important platform after win32 for id. From a biz standpoint it would be ludicrous to place it even on par with mac or os/2, but for our types of games that are designed to be hacked, linux has a big plus: the highest hacker to user ratio of any os. I don't personally develop on linux, because I do my unixy things with NEXTSTEP, but I have a lot of technical respect for it.
MacOS
From a money making standpoint, the only OS other than win32 that matters, and it doesn't matter all that much. We have professional ports done to MacOS instead of unsupported hack ports, which is a mixed blessing. They come out a lot later (still waiting for quake..), but are more full featured. I have zero respect for the MacOS on a technical basis. They just stood still and let microsoft run right over them from waaay behind. I wouldn't develop on it.
OS/2
A native OS/2 port of any of our products is unlikely. We just don't care enough, and we are unwilling to take time away from anything else.
SGI
I don't particularly care for IRIX as a development environment (compared to NT or NEXTSTEP), but SGI has the coolest hardware to run GL apps on. Safe to assume future IRIX ports, but its not exactly a top priority.
AIX / OSF / HPUX / SOLARIS
I wouldn't start a port to any of these, but if a trusted party (Zoid) wanted to do them, I probably wouldn't object.
BeOS
I bought a BeBox because I am a solid believer in SMP, and I like clean, from-scratch systems. I was left fairly non plussed by it. Yes, it is lean and mean and does a couple things better than any other OS I have seen, but I just don't see any dramatic advantages to it over, say, NEXTSTEP. Lion (the company doing the mac quake port) has a BeOS port of quake sort of working, and have my full support in releasing it, but it will be strictly an act of charity on their part, so don't expect too much.
Plan9
I spent a few months running Plan9. It has an achingly elegent internal structure, but a user interface that has been asleep for the past decade. I had an older version of quake dedicated server running on it (don't ask me for it - I lost it somewhere) and I was writing a civilized window manager for it in my spare time, but my spare time turned out to be only a couple hours a month, and it just got prioritized out of existance.
NEXTSTEP
My faviorite environment. NT and linux both have advantages in some areas, but if they were on equal footing I would choose NEXTSTEP hands down. It has all the power of unix (there are lots of things I miss in NT), the best UI (IMHO, of cource), and it just makes sense on so many more levels than windows. Yes, you can make windows do anything you want to if you have enough time to beat on it, but you can come out of it feeling like you just walked through a sewer.
In the real world, things aren't on equal footing, and I do most of my work on NT now. I hold out hope that it may not stay that way. If apple Does The Right Thing with rhapsody, I will be behind them as much as I can. NEXTSTEP needs a couple things to support games properly (video mode changing and low level sound access). If apple/next will provide them, I will personally port our current win32 products over.
If I can convince apple to do a good hardware accelerated OpenGL in rhapsody, I would be very likely to give my win NT machine the cold shoulder and do future development on rhapsody. (I really don't need Quickdraw3D evangelists preaching to me right now, thank you)
-----------------------------------------
John Carmack's .plan for Mar 23, 1997
-----------------------------------------
Someone ran into my F40 in the parking lot, then took off. Words cannot do justice to how I feel right now.
If anyone knows a tall white male in the dallas area that now has red paint and carbon fibre on their tan pickup truck, turn the bastard in!
-----------------------------------------
John Carmack's .plan for Apr 02, 1997
-----------------------------------------
The second generation QuakeWorld is out and live now.
We will probably release a couple bug fix versions over the next week or so as problems are reported.
Overall, I'm pleased with the results - I think I have delivered very solid improvements in game play. I certainly learned a lot along the way. If you have anything resembling a decent connection, you should be able to play a good game. A server with a 400+ ms ping and 10% packet loss is still not going to play all that great, but you should just avoid those.
The packet size is about as small as it is going to get for the general cases. Any more shrinkage will have to be game-specific compression, like the specialized nail update.
I can make doors and plats move smoothly, but it will take a good chunk more development. This will probably be done for quake 2.
I have it all set up to clip movement against other players during prediction, but I probably need a day or two to finish it. I'm not confidant that I'll get to that anytime soon, though.
I really want to get client side demo recording and more spectator mode options (see through player's eyes, chase cam, etc), but I just don't have the time right now.
The next major upgrade will be a quakeworld that can run in software and OpenGL modes. A verite module will come later.
This combination (QW networking and switchable rendering) will be the base that we move all of our Quake 2 work over to.
-----------------------------------------
John Carmack's .plan for Apr 04, 1997
-----------------------------------------
Ok, the current OpenGL code no longer scales the status bar and console. You can stop complaining now. The next release will be the consolidated rendering code for quakeworld. I'm not sure when I will be able to make a standalone version.
The consolidated quake will also be available on NT-alpha as well as x86. If you have a powerstorm-T card, glquake works pretty good. Glint and oxygen cards don't work well enough, but the normal quake software version should work fine. We may get a little bit of asm code written for the software version.
-----------------------------------------
John Carmack's .plan for Apr 08, 1997
-----------------------------------------
Technical note:
Instanced brush models are not going to be well supported in the next release of the quake engine. Instanced bmodels are models that are created as an entirely seperate map, like the health boxes and ammo boxes. We are not going to use any instanced bmodels in q2, but I will probably leave the drawing code intact for backwards compatability. You will NOT be able to movement clip against them as a bsp hull, however. You could still use a solid bounding box if you really have to have one, though.
Brush models built as part of the world, like doors and plats, will remain with full functionality.
Instanced bmodels never were lighted properly, and a lot of code gets simpler with this decision.
-----------------------------------------
John Carmack's .plan for Apr 09, 1997
-----------------------------------------
Would anyone complain if I took out lookstrafe and keyboard look modifier? I'm cleaning up code, and I don't know of anyone that ever uses those features.
update: ok, +klook has it's supporters. Anyone for lookstrafe?
-----------------------------------------
John Carmack's .plan for Apr 24, 1997
-----------------------------------------
The consolidated QuakeWorld client has been working pretty well. I've been playing deathmatch with it in GL mode for the past week. There are still a number of things to do on it, and I haven't been working on it for a while due to higher priority tasks. A lot of other non-graphics things have changed in the new architecture as well.
It is really cool to be able to switch between software and gl without even restarting the game. We will be testing Quake 2 extensively in GL and even doing some specific development for it. My current wild guess is that about 15% of quake 2 customers will run the OpenGL version (there will be several cards coming out this year that are fast enough, besides just 3dfx), so it is definately a factor in our decisions.
The verite renderer will still be supported in quake 2, but it won't have the special features of glquake. (it will still have it's own custom features like anti-aliasing, though)
There is a very cool new surprise feature for you in the next gl release :)
----
For the past several days I have been working on a new version of qbsp that should be dramatically more robust for "crazy" maps. Raven is aproaching completion on Hexen 2, and they have a couple problems in their maps, so this is my top priority.
I figured out something about CSG / BSP operations that had been kicking around in the back of my head for almost two years now. The seperate (and epsilon issue prone) CSG phase is not needed at all if you directly contruct a BSP tree from volumes instead of from polygons. I have that part working, but there is just so much work in the tools that getting the rest of the stuff working again is taking quite a lot of effort.
I will make another tool release when things calm down, but understandably that is about at the bottom of my priority list.
-----------------------------------------
John Carmack's .plan for Apr 28, 1997
-----------------------------------------
I'm sure you have all heard about the 3drealms / quake deal by now. It took a long time to get everything nailed down, but it should be worth it.
The "quake 2" terminology is a little confusing. They have all the quake / glquake / quakeworld stuff right now, but they will be picking up the quake 2 codebase after we finish it.
I'm quite excited about this - it should be a very complimentary arrangement. We would never have done a game like Duke at id, but there are many valid styles of design that are mutually exclusive. Todd and the rest of the Duke team are hard working developers with a pretty clear vision of what they want. It happens to be different than our vision, but the market is plenty big enough for both of them.
-----------------------------------------
John Carmack's .plan for May 06, 1997
-----------------------------------------
Brian Hook has been hired as our new programmer.
Brian wrote the glide API for 3dfx, worked on CosmoGL, and wrote a book on 3d programming that he is now horribly embarrased about.
-----------------------------------------
John Carmack's .plan for May 12, 1997
-----------------------------------------
I have gotten several emails speculating that there will now be a native glide port of quake. Here is the straight answer:
I have considered a glide port many times (especially now that the rendering code is in a dll), but I allways reach the conclusion that it wouldn't be justified.
On the plus side, it could get a 10%-15% speedup over the OpenGL driver without going through too many contortions. Primarily by saving transforms for the lightmap pass and doing tightly packed vertex arrays for the enemy models.
The big drawback is that every codepath that gets added holds back future innovation. Just having software and gl is a lot of work, and I have allready commited to verite support. This is a difficult point for some people to understand, but it is crucially important. The more places I need to rewrite a feature, the less likely I am to put it in. If I only had the opengl version to worry about, Quake 2 would be so much cooler..
-----------------------------------------
John Carmack's .plan for May 14, 1997
-----------------------------------------
As some of you may know, a port of Quake was demod at apple's WWDC. Here is the full info:
A couple weeks ago, I got an email saying: "Hey! We heard you are porting quake for WWDC!". I replied: "Uh, first I've heard of it.. I was planning on supporting Quake 2 on it late this year.."
Well, I stole some time and went ahead and did it (mostly last weekend - running tight!). I'm quite happy with how it turned out, and I'm glad it made it for the demos.
It is actually a port of the current research QuakeWorld-merging-into-Quake2 codebase, so it only plays network games at the moment.
It is running through 24 bit display postscript, and doesn't have the assembly language compiled in, so don't believe anyone that says it was running faster than under windows. It was a fast demo system. There is a good chance that it will be a bit faster then win32 when I am done with it, because the direct-to-screen API doesn't require all the locks and unlocks of Direct Draw, and the sound access will avoid the DirectSound penalties, but basically they should be the same.
98% of the support I need for games is present in rhapsody, and now that there is an existing game for it, the remaining decisions can be rationally guided.
I am still going to press the OpenGL issue, which is going to be crucial for future generations of games.
I am definately going to support Quake 2 on rhapsody. I may make a public release of the QuakeWorld demo, but I will probably wait until we get the full screen api working. Omnigroup has a little qspy-like openstep program that we can use with it.
-----------------------------------------
John Carmack's .plan for May 22, 1997
-----------------------------------------
Bad news.
It looks like this is when "unsupported" really becomes unsupported
Glquake and QuakeWorld were fun to do, but keeping the datasets compatable with quake 1 really has held me back a lot. I badly wanted to get one more release out, but circumstances have forced me to finally ireversibly break with compatability, and I just don't have the time to devote any effort to a stagnant codebase. You probably wont see any more releases from Id until hexen 2 ships. Sorry.
I have given Zoid and Jack Mathews free license to extend and upgrade the QuakeWorld codebase from the last released revision, so this may actually mean that QW receives more attention than I was able to give it.
On the bright side, the new bsp format will offer some great new capabilities that will be apreciated by all:
Greater robustness. Only one bsp tree is built, and no surfaces are generated that weren't part of the map brushes.
No fixed clipping hull restrictions. You can now set any mins/maxs you like.
You can tell the texture that a trace clips on in the game, so different surface attributes are now possible.
Textures are no longer stored in the bsp file.
Full color lightmaps for glquake. The "surprise" that I mentioned before was colored lighting hacked into glquake in a way that didn't require a change in the format, but this is better.
If any hard-core add on hackers can present a serious case for additional modifications to the bsp file, now is the time to let me know.
-----------------------------------------
John Carmack's .plan for Jun 19, 1997
-----------------------------------------
I'm pretty damn pleased with things right now.
We are just buttoning up the E3 demo stuff, and it looks really good. It is clearly way alpha meterial, but people should be able to project where it is going.
The timing is a bit inconvenient for us, because we still aren't quite through with converting all the .qc work that Cash did over to straight C code in the new engine. The monsters are just barely functional enough to show, with none of the new behavior in. If E3 was a week or two later, the demos would almost be real playtesting.
Q2 is going to be far and away the highest quality product id has ever done. There are new engine features, but the strength of the product is going to be how everything is fitted together with great care. (don't worry, next year will be radical new technology all over again)
--
Sound is being improved in a number of ways.
All source samples are 22 khz / 16 bit, and you can restart the sound system for different quality levels without exiting the game. high quality sound will require more memory than the base 16 meg system. The system can automatically convert to 11 khz / 8 bit sounds, but we are probably going to include a seperate directory with offline converted versions, which should be slightly higher quality. Homebrew paatches don't need to bother.
Sounds can now travel with a moving object. No dopler effects, but it positions properly. (well, spatialization is a bit fucked this very instant, but not for long)
I finally got around to tracking down the little bug with looping sounds causing pops.
I have intentions to do three more things with the sound engine, but the realistic odds are that they won't all make it in:
Voice over network. I definately don't have time to do a super-compressed version, but I can probably hack something in that the T1 players would have fun with.
Radiosity sound solution. Its obvious in retrospect, but it was a "eureka!" thought for me when I realized that the same functions that govern the transport of light for radiosity also apply to sound. I have research plans for next-generation technology that include surface reflection spectrums and modeling the speed of sound waves, but I think I can get a simplified solution into Q2 to provide an ambient soundscape with the same level of detail as the lightmaps. I'm a little concerned about the memory footprint of it, but I'm going to give it a shot.
Syncronized, streaming sound from disk. Special events and movie demos won't need to precache gigantic sounds, and they can rely on the timing.
--
Q2 has a generalized inventory structure and status display that should be adaptable to just about anything anyone wants to do in a TC.
--
On saturday, I give my 328 away at E3. I know that there were lots of issues with the contest, and to be honest, I probably wouldn't have done the nationwide contest if I could have forseen all the hassle (I could have just given it away at #quakecon..), but the finals should still be really cool. It just wasn't possible to make the contest "completely fair". Not possible at all. In any case, I don't think anyone will deny that the finalists are some of the best quake players around.
--
I doubt I can convey just how well things are going here. Things probably look a little odd from the outside, but our work should speak for itself. I have been breaking into spontanious smiles lately just thinking about how cool things are (of course, that could just be a sleep deprivation effect..).
We have a totally kick-ass team here.
We are on schedule. (no shit!)
We are doing a great product.
Everyone watch out!
-----------------------------------------
John Carmack's .plan for Jun 22, 1997
-----------------------------------------
Ok, I'm finally updating my .plans at the top like everyone else..
E3 was interesting, and the tournement went extremely well.
You would think that the top deathmatchers would be an evenly matched group, seperated by mere milliseconds of response time, and the matches would be close.
Its not like that at all. There are master players. And there is Thresh.
We were watching him play with our jaws hanging open. I don't think he was killed a single time in the finals. He did things we had never seen before. It was amazing to watch.
I feel a lot better about the contest now, because even if the sixteen finalists weren't necessarily the sixteen best players due to internet issues, I do think that the grand prize winner IS the best single player.
The level of sportsmanship was gratifying, especially given the stakes. No sore losers, no tantrums. Everyone was cool.
After the finals, a japanese champion (highroller) asked for a match with Thresh. I expected him to pass, considering the pressure of the tournement, but he happily accepted, and delivered an eighty-something to negative-three beating (which was accepted with good grace).
I don't see much point to any more big tournements until a few more of these mutant superpowered deathmatchers show up..
As far as everything else at E3 goes, I saw a bunch of good looking games, but I am fairly confident of two things:
Nobody is going to eclipse Quake 2 this christmas. Different tradeoffs are being made that will appeal to different people, and there are going to be other products that are at least playing in the same league, but Q2 should be at the top of the pile, at least by the standards we judge games. Several licensees will be picking up all the Q2 features for their early '98 products, so games should get even better then. (ok, I guess that is just my cautious, long-winded way of saying Q2 will rule..)
Some notable companies are going to ship longer after us than they are expecting to, or make severe compromises. I wouldn't advise holding your breath waiting for the quoted release dates. Relax, and let the developers get things out in due time.
Ugh. I haven't coded in three days. Withdrawal.
-----------------------------------------
John Carmack's .plan for Jun 25, 1997
-----------------------------------------
We got the new processors running in our big compute server today.
We are now running 16 180mhz r10000 processors in an origin2000. Six months ago, that would have been on the list of the top 500 supercomputing systems in the world. I bet they weren't expecting many game companies. :)
Some comparative timings (in seconds):
[code]
mips = 180 mhz R10000, 1meg secondary cache
intel = 200 mhz ppro, 512k secondary cache
alpha = 433 mhz 21164a, 2meg secondary cache
qvis3 on cashspace:
cpus mips intel alpha
---- ---- ---- ----
1 608 905 470
2 309 459
3 208 308
4 158 233
8 81
12 57
16 43
[/code]
(14 to 1 scalability on 16 cpus, and that's including the IO!)
The timings vary somewhat on other tools - qrad3 stresses the main memory a lot harder, and the intel system doesn't scale as well, but I have found these times to be fairly representative. Alpha is almost twice as fast as intel, and mips is in between.
None of these processors are absolutely top of the line - you can get 195 mhz r10k with 4meg L2, 300 mhz PII, and 600 mhz 21164a. Because my codes are highly scalable, we were better off buing more processors at a lower price, rather than the absolute fastest available.
Some comments on the cost of speed:
A 4 cpu pentium pro with plenty of memory can be had for around $20k from bargain integrators. Most of our Quake licensees have one of these.
For about $60k you can get a 4 cpu, 466 mhz alphaserver 4100. Ion Storm has one of these, and it is twice as fast as a quad intel, and a bit faster than six of our mips processors.
That level of performance is where you run into a wall in terms of cost.
To go beyond that with intel processors, you need to go to one of the "enterprise" systems from sequent, data general, ncr, tandem, etc. There are several 8 and 16 processor systems available, and the NUMA systems from sequent and DG theoretically scale to very large numbers of CPUS (32+). The prices are totally fucked. Up to $40k PER CPU! Absolutely stupid.
The only larger alpha systems are the 8200/8400 series from dec, which go up to 12 processors at around $30k per cpu. We almost bought an 8400 over a year ago when there was talk of being able to run NT on it.
Other options are the high end sun servers (but sparc's aren't much faster than intel) and the convex/hp systems (which wasn't shipping when we purchased).
We settled on the SGI origin systems because it ran my codes well, is scalable to very large numbers of processors (128), and the cost was only about $20k per cpu. We can also add Infinite Reality graphics systems if we want to.
Within a couple years, I'm sure that someone will make a plug-in SCI board for intel systems, and you will be able to cobble together NUMA systems for under $10k a cpu, but right now the SGI is the most suitable thing for us.
I have been asked a few times if Quake will ever use multiple processors. You can allways run a dedicated server on one cpu and connect to it to gain some benefit, but that's not very convenient, doesn't help much, and is useless for internet play.
It's waaaay down on the priority list, but I have a cool scheme that would let me make multiple copies of the software rendering dll and frame pipeline the renderers. Response is cut by half and the frame rate would double for two cpus, but pipelining more than a frame would be a bad idea (you would get lag on your own system).
I wouldn't count on it, but some day I might take a break from serious work and hack it in.
There is no convenient way to use multiple processors with the hardware accelerated versions, except to run the server on a seperate cpu.
That will probably be an issue that needs to be addressed in the lifespan of the next generation technology. Eventually people are going to start sticking multiple cpus (or multiple thread issue systems sharing resources) on a single chip, and it will become a consumer level item. I'm looking forward to it.
-----------------------------------------
John Carmack's .plan for Jul 03, 1997
-----------------------------------------
This little note was issued to a lot of magazines by microsoft recently. Just for the record, they have NOT contacted us about any meetings.
All the various dramas in this skit haven't quite settled down, but it looks like microsoft is going to consciously do The Wrong Thing, because of politcal issues. Sigh.
Our goal was to get the NT OpenGL MCD driver model released for win-95, so IHVs could easily make robust, high performance, fully compliant OpenGL implementations. Microsoft has squashed that. Flushed their own (good) work down the toilet.
The two remaining options are to have vendors create full ICD opengl implementations, or game specific mini-drivers.
Full ICD drivers are provided by intergraph, 3dlabs, real3d, and others, and can run on both NT and 95 (with code changes). Microsoft still supports this, and any vendor can create one, but it is a lot of work to get the provided ICD code up to par, and bug prone. On the plus side, non-game tools like level editors can take full advantage of them.
Minidrivers certainly work fine - we have functional ones for 3dfx and powerVR, and they have the possibility of providing slightly better performance than fully compliant drivers, but partial implementations are going to cause problems in the future.
We will see some of both types of drivers over the next year, and Quake 2 should work fine with either. We also intend to have Quake 2 show up on several unix systems that supports OpenGL, and I still hope that rhapsody will include OpenGL support (we'll probably port a mini-drivers if we can't get real support).
Once again, we won't be idiotic and crusade off a cliff, but we don't have to blindly follow microsoft every time they make a bad call.
Subject: Microsoft D3D vs. OpenGL
Author: Julie Whitehead at Internet
Date: 6/23/97 10:01 AM
Dear Editor,
You may be aware of a press release that was issued On June 12, by Chris Hecker, former MS employee and developer of D3D [sic]. The statement asks Microsoft to develop a stonger link between D3D and OGL.The press release, was signed by several game developers representing the top tier 3-D game developers. Microsoft is dedicated to maintaining an active relationship with its DirectX developers. In response to this request Microsoft will host the developers included in the statement at a developers roundtable in July. The purpose of the roundtable is to openly consolidate input and feedback from developers. Tentative date for the roundtable is immediately following Meltdown, July 18.
Direct3D is Microsoft's recommended API for game developers with more than 100 developers using Direct3D as the defacto consumer API. OGL is widely regarded as a professional API designed for high precision applications such as CAD, CAM, etc. Our hope is that this round table will provide Microsoft with the feedback required to evolve our 3D APIs in a way that delivers the best platform for our developers.
If you have any questions or wish to speak with a Microsoft spokesperson, please let me know.
Julie Whitehead
-----------------------------------------
John Carmack's .plan for Jul 07, 1997
-----------------------------------------
The quality of Quake's software has been a topic of some discussion lately. I avoid IRC like the plague, but I usually hear about the big issues.
Quake has bugs. I freely acknowledge it, and I regret them. However, Quake 1 is no longer being actively developed, and any remaining bugs are unlikely to be fixed. We would still like to be aware of all the problems, so we can try to avoid them in Quake 2.
At last year's #quakecon, there was talk about setting up a bug list maintained by a member of the user community. That would have been great. Maybe it will happen for Quake 2.
The idea of some cover up or active deception regarding software quality is insulting.
To state my life .plan in a single sentance: "I want to write the best software I can". There isn't even a close second place. My judgement and my work are up for open criticism (I welcome insightfull commentary), but I do get offended when ulterior motives are implied.
Some cynical people think that every activity must revolve around the mighty dollar, and anyone saying otherwise is just attempting to delude the public. I will probably never be able to convince them that isn't allways the case, but I do have the satisfaction of knowing that I live in a less dingy world than they do.
I want bug free software. I also want software that runs at infinite speed, takes no bandwidth, is flexible enough to do anything, and was finished yesterday.
Every day I make decisions to let something stand and move on, rather than continuing until it is "perfect". Often, I really WANT to keep working on it, but other things have risen to the top of the priority que, and demand my attention.
"Good software" is a complex metric of many, many dimensions. There are sweet spots of functionality, quality, efficiancy and timeliness that I aim for, but fundamentally YOU CAN'T HAVE EVERYTHING.
A common thought is that if we just hired more programmers, we could make the product "better".
It's possible we aren't at our exactly optimal team size, but I'm pretty confidant we are close.
For any given project, there is some team size beyond which adding more people will actually cause things to take LONGER. This is due to loss of efficiency from chopping up problems, communication overhead, and just plain entropy. It's even easier to reduce quality by adding people.
I contend that the max programming team size for Id is very small.
For instance, sometimes I need to make a change in the editor, the utilities, and the game all at once to get a new feature in. If we had the task split up among three seperate programmers, it would take FAR longer to go through a few new revs to debug a feature. As it is, I just go do it all myself. I originated all the code in every aspect of the project, so I have a global scope of knowledge that just wouldn't be possible with an army of programmers dicing up the problems. One global insight is worth a half dozen local ones.
Cash and Brian assist me quite a lot, but there is a definite, very small, limit to how many assistants are worthwhile. I think we are pretty close to optimal with the current team.
In the end, things will be done when the are done, and they should be pretty good. :)
A related topic from recent experience:
Anatomy of a mis-feature:
----
As anyone who has ever disected it knows, Quake's triangle model format is a mess. Any time during Quake's development that I had to go back and work with it, I allways walked over to Michael and said "Ohmygod I hate our model format!'. I didn't have time to change it, though. After quake's release, I WANTED to change it, especially when I was doing glquake, but we were then the proud owners of a legacy data situation.
The principle reason for the mess is a feature.
Automatic animation is a feature that I trace all the way back to our side-scroller days, when we wanted simple ways to get tile graphics to automatically cycle through animations without having to programatically each object through its frames.
I thought, "Hmm. That should be a great feature for Quake, because it will allow more motion without any network bandwidth."
So, we added groups of frames and groups of skins, and a couple ways to control the timing and syncronization. It all works as designed, but parsing the file format and determining the current frames was gross.
In the end, we only used auto-frame-animation for torches, and we didn't use auto-skin-animation at all (Rogue did in mission pak 2, though).
Ah well, someone might use the feature for something, and its allready finished, so no harm done, right?
Wrong. There are a half dozen or so good features that are apropriate to add to the triangle models in a quake technology framework, but the couple times that I started doing the research for some of them, I allways balked at having to work with the existing model format.
The addition of a feature early on caused other (more important) features to not be developed.
Well, we have a new model format for Quake 2 now. Its a ton simpler, manages more bits of precision, includes the gl data, and is easy to extend for a couple new features I am considering. It doesn't have auto-animation.
This seems like an easy case - almost anyone would ditch auto-animation for, say, mesh level of detail, or multi-part models. The important point is that the cost of adding a feature isn't just the time it takes to code it. The cost also includes the addition of an obsticle to future expansion.
Sure, any given feature list can be implemented, given enough coding time. But in addition to coming out late, you will usually wind up with a codebase that is so fragile that new ideas that should be dead-simple wind up taking longer and longer to work into the tangled existing web.
The trick is to pick the features that don't fight each other. The problem is that the feature that you pass on will allways be SOMEONE's pet feature, and they will think you are cruel and uncaring, and say nasty things about you.
Sigh.
Sometimes the decisions are REALLY hard, like making head to head modem play suffer to enable persistant internet servers.
-----------------------------------------
John Carmack's .plan for Jul 11, 1997
-----------------------------------------
Zoid commented that my last .plan update sounded like Fred Brooks "The Mythical Man-Month". He is certainly correct.
When I read TMMM two years ago, I was stunned by how true and relevent it was. I have something of a prejudice against older computer books - I think "If its more than a five years old, it can't be very relevent" (sure, thats not too rational, but what prejudice is?).
Then I go and read this book that is TWENTY YEARS old, that talks about experience gained IN THE SIXTIES, and I find it mirroring (and often crystalizing) my thoughts on development as my experiences have taught me.
It even got me fired up about documenting my work. For about a day :)
I had to fly out to CA for biz on thursday, so I decided to grab and re-read TMMM on the plane.
It was just as good the second time through, and two more years of development under my belt hasn't changed any of my opinions about the contents.
If you program (or even work around software development), you should read this book.
-----------------------------------------
John Carmack's .plan for Jul 25, 1997
-----------------------------------------
Id Software went to the drag strip today.
The 100 degree heat was pretty opressive, and my NOS regulator wasn't working, but a good time was had by all.
I made six runs in the 126 to 133 mph range and didn't even burn a spark plug, which is a nice change from a couple road track events I have been to.
Best times for everyone:
[code]
Bob Norwood's PCA race car: 10.9 / 133 mph (slicks)
My turbo testarossa 12.1 / 132
Adrian's viper 13.5 / 105
Todd's 'vette 13.9 / 101
Tim's porsche 14.3 / 96
Bear's supra: 14.4 / 96
Cash's M3 15.2 / 94
[/code]
My TR is never going to be a good drag car (>4000 lbs!), but when we go back on a cool day this fall and I get my NOS running, it should be good for over 140 in the quarter. 50 mph to 200 mph is it's real sweet spot.
I think Bear is heading for the chip dealer so he can get ahead of Tim :)
-----------------------------------------
John Carmack's .plan for Jul 30, 1997
-----------------------------------------
quake2 +set maxclients 200
:)
The stage is set for ultra-large servers. Imagine everyone at QuakeCon in one gigantic level! A single T1 could run 80 internet players if it wasn't doing anything else, a switched ethernet should be able to run as many as we are ever likely to have together in one place.
There will be a number of issues that will need to be resolved when this becomes a reality, but the fundamentals are there.
There will probably be issues with UDP packet dropping at the ethernet card level that will need to be worked around with a seperate qued thread.
Quake 2 isn't as cpu intensive as QuakeWorld, but I'm not sure even a Pentium-II 300 could run 200 users. An alpha 21264 could certainly deal with it, though.
The new .bsp format has greatly increased size limits, but you could still make a map that hits them. The first one to be hit will probably be 64k brush sides. Ten thousand brushes can make a really big level if you don't make it incredibly detailed. Loading a monster map like that will probably take over a minute, and require 32+ megs of ram.
I should probably make an option for death messages to only be multicast to people that are in the potentially hearable set, otherwise death messages would dominate the bandwidth.
Everyone should start thinking about interesting rules for huge games. A QuakeArmies dll has immense potential. Enemy lines, conquering teritory, multiple clan bases, etc.
Cooperating servers will be possible with modified dlls, but I probably won't include any specific code for it in the default game.dll.
-----------------------------------------
John Carmack's .plan for Aug 05, 1997
-----------------------------------------
At siggraph
* fix qe4 autosave
* merged qlumpy into qdata, save seperate files
* changed quaked to use texture directories
+ fix leaktest option
+ show texture directory on inspector window
+ show full texture name somewhere when clicked on
+ texture info overrides
remap maps to share common textures?
-----------------------------------------
John Carmack's .plan for Aug 06, 1997
-----------------------------------------
* qe4 texture directories
* fixed vid_restart
* hacked alpha colors for cards without src*dst
* fixed qdata vc compiler bug in arg parsing
* qe4 surface inspector
-----------------------------------------
John Carmack's .plan for Aug 07, 1997
-----------------------------------------
+ add animation frames to bsp file texinfos
- make bmodel frames just add to texinfo?
- should msurface flags hold the texinfo flags?
+ make window content implicit if any surfaces are trans
+ CONTENTS_ORIGIN flag
+ nodetail bsp
+ select face option in qe4
+ use monsterclips!
+ gl fullbright textures are still 2x brightness
moveable alpha surfaces
merge find texture dialog into surface inspector
fix qdata unix directory stuff
get rid of mod->skins, use mod->images
-----------------------------------------
John Carmack's .plan for Aug 08, 1997
-----------------------------------------
* added origin brush support to old bsp for raven
+ add edge planes for brush hulls
- rate is broken - inventory fix
-----------------------------------------
John Carmack's .plan for Aug 09, 1997
-----------------------------------------
* combined bsp tools into a single vc project
* new texture animation solution
* make any com_error drop the loading plaque
* tools and quake2 work with new bsp format
+ combine project files of bsp tools
+ anything translucent is automatically a detail contents
- duplicate texinfo for animations?
+ store out contents from trace!
+ arbitrary visleafs mappings
+ scanmaps option for pak file building of textures
+ delta lightstyle controls from server
+ max moveleafs problem
+ make r_dowarp a server passed variable?
+ why is hunk_begin different in software?
don't forget to set SURF_NOSUBDIV on warps and sky!
compress ff in visdata as well as 0?
trinity idea: model light haze around every emiter
trinity idea: allways model volumetric lights by rendering back sides
CONTENTS_VOLUME
do a wavy specular water novelty
allow arbitrary chained lightmaps on a surface?
game.dll controlable particles
get rid of SURF_PLANEBACK
player sounds when moving? (breathing / footsteps / hitting walls)
rename .bsp to .bs2 ?
high frame rate run turn chunkiness
-----------------------------------------
John Carmack's .plan for Aug 10, 1997
-----------------------------------------
I went to siggraph last monday to give a talk about realtime graphics for entertainment.
The only real reason I agreed to the talk (I have turned down all other offers in the past) was because Shigeru Miyamoto was supposed to be on the panel representing console software. Id software was really conceived when me, Tom, and Romero made a Super Mario 3 clone after I figured out how to do smooth scrolling EGA games. We actually sent it to nintendo to see if they wanted to publish a PC game, but the interest wasn't there. We wound up doing the Commander Keen games for Apogee instead, and the rest is history.
I was looking forward to meeting Mr. Miyamoto, but he wound up canceling at the last minute. :(
Oh well. I hope everyone that went enjoyed my talk. All the other speakers had powerpoint presentations and detailed discussion plans, but I just rambled for an hour..
I notced that there was a report about my discussion of model level of detail that was in error. I have an experimental harness, an algorithm, and a data structure for doing progressive mesh style LOD rendereing in the quake engine, but I suspect it won't make it into the production Quake 2. Other things are higher priority for us. I may assist some of the quake licensees if they want to pursue it later.
-
A couple data / feature changes going into the latest (and I hope final) revision of the Quake bsp file format:
Back in my update a month ago where I discussed losing automatic frame animation in models to clean up the format and logic, I mentioned that I still supported automatic texture animation.
Not anymore. There were several obnoxious internal details to dealing with it, especially now with textures outside the bsp file, so I changed the aproach.
When a texture is grabbed, you can now specify another texture name as the next animation in a chain. Much better than the implicit-by-name specification from Quake1.
No animation is automatic now. A bmodel's frame number determines how far along the animation chain to go to find the frame. Textures without animation chains just stay in the original frame.
There is a slight cost in network traffic required to update frame numbers on otherwise unmoving objects, but due to the QuakeWorld style delta compression it is still less than a Quake 1 scene with no motion at all.
The benefit, aside from internal code cleanliness, is that a game can precisely control any sequence of animation on a surface. You could have cycles that go forward and backwards through a sequence, you could make slide projectors that only change on specific inputs, etc.
You could not independantly animate two sides of a bmodel that were not syncronized with the same number of frames, but you could allways split it into multiple models if your really needed to.
Everything is simple when its done, but I actually agonized over animation specification for HOURS yesterday..
The last significant thing that I am working on in the map format is leaf clustering for vis operations. You can specify some map brushes as "detail" brushes, and others as "structural" brushes. The BSP and portal list is built for just the structural brushes, then the detail brushes are filtered in later.
This saves a bit of space, but is primarily for allowing complex levels to vis in a reasonable amount of time. The vis operation is very sensitive to complexity in open areas, and usually has an exponentially bad falloff time. Most of the complexity is in the form of small brushes that never really occlude anything. A box room with ten torch holders on the walls would consist of several dozen mostly open leafs. If the torch holders were made detail brushes, the room would just be a single leaf.
A detail / structural seperation is also, I believe, key to making a portal renderer workable. I had a version of Quake that used portals at the convex volume level, and the performance characteristics had considerably worse-than-linear falloff with complexity. By reducing the leaf count considerably, it probably becomes very workable. I will certainly be reevaluating it for trinity.
* trans33, trans66, flow flags in gl
* damped warp modulation in gl
* ref_soft running with new data
+ shots are exploding on the sky again
+ auto set window contents if translucent
+ don't set qe4 texture unless notexture
+ try new console background
+ finish animation cycling
detail brushes could be extended to be destroyable
new texture specification by three points?
check -tmpin -tmpout in bsp utils
rename texinfo to surfinfo?
pitch change during jumping
minimized window notification when a new client joins?
should origin brushes be included in bsp file for completeness?
use nodraw flag
pitch change when ducking
qrad light bleeds
-----------------------------------------
John Carmack's .plan for Aug 11, 1997
-----------------------------------------
* don't set qe4 texture unless notexture
* don't set qe4 texture on cancel unless changed
* grabbed new menu and console
* invert mouse off in default.cfg
* all software flags
* mist contents
+ imagelist command in software
trinity: save out projection outlines from editor for textures
add a 5th control axis (and 6th?) for spaceorb ducking
gl: don't keep lightmap blocks around in main memory?
entities not visible (or only visible) to owners
look in direction other than motion for hmd
quake as root directory problem
dir command
software surface / edge allocation issues
-----------------------------------------
John Carmack's .plan for Aug 12, 1997
-----------------------------------------
* qe4 project on command line
* qe4 rshcmd replacement
* qe4 select face
* qe4 avoid multiple autosaves
* qe4 region selected brushes
* bindlist command
* imagelist command in ref_soft
+ leaktest
+ load game.dll from gamedir
pendulum motion
no jump on lava floor?
-game
16 bit wall textures
-----------------------------------------
John Carmack's .plan for Aug 13, 1997
-----------------------------------------
* cls.fixedimage support
* no frame before cinematic fix
* menu during cinematic fix
+ ingame cinematic state
+ indemo cinematic state
- move fraglogfile into game dll
+ layout language beyond simple centerprint
+ killserver needs to kill demos as well
+ must kill cinematic after menu, or restart palette
+ disconnected can be either at a console or running the demo + intro cinematic
needs to be part of the game
force nolerp lag?
put ip filtering in game dll
handle localmodels explicitly, rather than as *num
don't send heartbeats if not running a network game?
move viewmodel for all accelerations, including jumping and landing
fade out centerprints
design quit screen to allow addons to get credits
be consistant with window title bars
mp3 audio
qe4: downsample option, nomipmap option
-----------------------------------------
John Carmack's .plan for Aug 14, 1997
-----------------------------------------
* qe4 project dialog fix
* intermission spots and movement
* hud transfer framework
+ micro levels that just play cinematics?
+ BUTTON_ANY option
remove oldorigin
use static arrays for map elements in renderers?
unit level statistics
-----------------------------------------
John Carmack's .plan for Aug 15, 1997
-----------------------------------------
* smart background clear
* worked around 100% viewsize floating point bug
* increased base surface cache size
* unified server commands and prog commands
* fixed same level reload bug in ref_soft
+ are lightmaps allways being saved, even if all black?
+ is notify box used for anything?
+ toggleconsole when connected to a net game
+ server needs to be able to send staticpics
+ draw to front buffer without swapping option
- can game.dll register commands?
+ direct sound and keyboard restart so ref can destroy window
+ loading plaque on local system doesn't wait for hud erase
frame flicker option for evaluating missed VBL?
way to add remote commands on client side by scripts?
check client entering during intermission
moveable transparant bmodels
use sneaking in shadows to let players get a good look at more monsters
translate cinematic to greyscale instead of blanking?
remove zombietime?
are userinfo updates getting sent?
-----------------------------------------
John Carmack's .plan for Aug 16, 1997
-----------------------------------------
download static screen images?
+ how to change semi-protected variables without killing server?
+ how do demo servers progress to the next server?
+ how does the client distinguish between a demo server? parm to map command?
+ demo servers have special console behavior and don't warn on game restart
+ do not allow remote connects to a demo server
+ no loading plaque if fullcon
+ cinematic trailing pic for victory
+ demo view angles?
+ text before next level after completed
+ replace draw_beginrawscene with setpalette?
+ keys should go to game when running cinematic, not console
+ does the console key serve as a skip-demo key on real servers?
+ need to flag unit ends for stats, vs simple transfers
+ pause demos and cinematics while menu is up
visible que on players when typing and when lagged?
make sure there is never a server running if client is fullcon
must force full graphics reload on game change
don't require full precache of all weapons?
demo servers won't work for remote connections, because packets can be dropped
prevent map / demomap changes without killserver
map demo1.dem during game messes up
victory freeze
-----------------------------------------
John Carmack's .plan for Aug 17, 1997
-----------------------------------------
* demo angles
* fixed initial lightmap cache value
* disconnect now does an ERR_DROP to kill server as well
* button_any support
* bad fov problem
aliaslist
never nextserver on finale
blaster autorepeat problem
end cinematic loading flicker
-----------------------------------------
John Carmack's .plan for Aug 18, 1997
-----------------------------------------
I get asked about the DOOM source code every once in a while, so here is a full status update:
The Wolfenstein code wasn't much of a service to release - it was 16 bit dos code, and there wasn't much you could do with it. Hell, I don't think it even compiled as released.
The DOOM code should be a lot more interesting. It is better written, 32 bit, and portable. There are several interesting projects that immediately present themselves for working with the code. GLDOOM and a packet server based internet DOOM spring to mind. Even a client/server based DOOM server wouldn't be too hard to do.
I originally intended to just dump the code on the net quite some time ago, but Bernd Kreimeier offered to write a book to explain the way the game works. There have been a ton of issues holding it up, but that is still the plan. If things aren't worked out by the end of the year, I will just release things in a raw form, though.
My best case situation would be to release code that cleanly builds for win32 and linux. Bernd is doing some cleanup on the code, and some of the Ritual guys may lend a hand.
One of the big issues is that we used someone else's sound code in dos DOOM (ohmygod was that a big mistake!), so we can't just release the full code directory. We will probably build something off of the quake sound code for the release.
I think I am going to be able to get away with just making all the code public domain. No license, no copyleft, nothing. If you apreciate it, try to get a pirate or two to buy some of our stuff legit..
* lightmap building errors
+ qe4: build in detail mode
+ animating textures
+ no different quantities on items?
+ target_secretcounter
the inherent problems of simplicity by complexity
-----------------------------------------
John Carmack's .plan for Aug 19, 1997
-----------------------------------------
* leaktest
+ min clamp extents
-----------------------------------------
John Carmack's .plan for Aug 23, 1997
-----------------------------------------
* cluster code
- boxcontents?
+ dump rgb lightmaps for software?
+ alias model aspect ratios different in software and gl?
share data between cmodel and ref
triangulate mightsee on vis?
malloc all cmodel arrays?
DONT_PRECACHE flag for player weapons?
make an ERR_DISCONNECT that doesn't print ERROR: ?
don't load entire map twice in cmodel and ref!
show clusterviscount for bsp time optimizations?
server/client communication for skin overrides
-----------------------------------------
John Carmack's .plan for Aug 24, 1997
-----------------------------------------
* qe4 slow startup
* qrad
+ detail clip brushes?
+ extra brush clip planes
change qdata colormap to not use 0 and 255 for win palette
-----------------------------------------
John Carmack's .plan for Aug 25, 1997
-----------------------------------------
I want to apologize for some of the posturing that has taken place in .plan files.
I have asked that attacks on our competition no longer apear in .plan files here. I don't think it is proper or dignified.
If everyone clearly understood that an individual's opinion is only that - the opinion of a single individual, I wouldn't have bothered. Unfortunately, opinions tend to be spread over the entire group, and I am not confortable with how this makes me perceived.
Building up animosity between developers is not a very worthwhile thing.
A little chest-beating doesn't really hurt anything, but putting down other developers has negative consequences.
I think that we have a track record that we can be proud of here at id, but we are far from perfect, and I would prefer to cast no stones.
The user community often exerts a lot of pressure towards confrontation, though. People like to pick a "side", and there are plenty of people interested in fighting over it. There are a lot of people that dislike id software for no reason other than they have chosen another "side". I don't want to encourage that.
Magazine articles are usually the trigger for someone getting upset here. Its annoying to have something you are directly involved in misrepresented in some way for all the world to see. However, I have been misquoted enough by the press to make me assume that many inflamatory comments are taken out of context or otherwise massaged. It makes a good story, after all.
Sure, there ARE developers that really do think they are going to wipe us off the face of the earth with their next product, and don't mind telling everyone all about it. Its always possible. They can give it their best shot, and we'll give it ours. If they do anything better, we'll learn from it.
* fixed water bsp bug! yeah!
* new tools in production
* view pitching with running
* weapon turn lagging
* debug graph
+ screen sizedown is not clearing again
+ animating textures
+ weapon change sounds should be server side
+ QE4: surface inspector apply is slow
qe4: seperate "make current texture" from "make and set all"
currentmodel name problem in gl_model
userinfo changes
-----------------------------------------
John Carmack's .plan for Aug 26, 1997
-----------------------------------------
* debuggraph on top
* better bobtime / bobcycle
* face seperation overrun bug
* fast surface dialog
* show detail on camera title
+ link commands for playing from the cd
qe4: view menu checkboxes are wrong
-----------------------------------------
John Carmack's .plan for Aug 27, 1997
-----------------------------------------
* fixed off-by-one cluster count
* fixed surf/content bit mismatch
* gun bob cycles
* falling pitch change
- make a fat pvs for rendering?
+ trace needs to return contents
+ fix BUTTON_ANY
+ rendering beams
+ delta lightstyle controls from server
+ finish animation cycling
QE4: deleting brushes doesn't count as modified?
initial time on spawn Tent
underwater caustics
make all bobbing effects cvars
title on inspector is broken for textures
moveable alpha surfaces
don't forget to set SURF_NOSUBDIV on warps and sky!
freeze map just sets a HUD of the victory screen
server scoreboard
-----------------------------------------
John Carmack's .plan for Aug 28, 1997
-----------------------------------------
* fixed entity culling on gl
* sorted axial and edge bevels on all brushes
+ entity culling in GL
+ imagelist should have the downsampled sizes
+ software should dump rgb lightmap data
an origin brush will never change a texinfo?
NO! the offsets can change
are brush numbers messed up because of removed brushes?
plat push into floor
use textureisresident in imagelist?
load mip levels seperately
duplicate planes
make set detail not work on entities
trinity: pivot feet! general atmospherics!
ray trace: texture+s/t for each sample, hardware reconstructs
walk up stairs by slope hitches up
animating textures
QE4: use gentextures
QE4: flush all textures option
-----------------------------------------
John Carmack's .plan for Aug 29, 1997
-----------------------------------------
sick
-----------------------------------------
John Carmack's .plan for Aug 30, 1997
-----------------------------------------
* changed snapnormal
* fixed BUTTON_ANY
* unix makefile
* pic server
* runcinematic call
* console over cinematic fix
+ console key during game cinematics
+ version number for quake 2?
+ cinematic set palette needs to clear screen
use cluster level leafs for sound radiosity
jittered texel centers?
trinity: continuous textures, surface cache on all
make net, pause, and drawcenterstring HUDs
-----------------------------------------
John Carmack's .plan for Aug 31, 1997
-----------------------------------------
* fixed trigger_always
* game dll by search path
* cinematic NULL bug
* help computer
+ get rid of datablocks?
+ dll init must clear the persistant data
- savegame needs to save game.dll name
+ save directory?
+ put pcx_t into qfiles.h?
+ unify all hud work into g_hud.c
set command with flags?
should "path" be renamed to pathlist?
trigger_allways should be fixed size
gameversion
somehow don't resend big deltas (scoreboard hud) until ack?
client side feedback
glgentextures
-----------------------------------------
John Carmack's .plan for Sep 01, 1997
-----------------------------------------
* QE4: bug with texture attributes on non-visible surfaces
* stack bug for initial light maps
+ pink lightmaps?
+ alt-tab should only minimize if full screen
+ version as command and var?
splashing sound when swimming at surface?
brains make view roll around
wasted polygons outside maps
vis decompression overruns?
INVISIBLE_TO_OWNER INVISIBLE_TO_OTHER
make * model names visible
GL_MipMap overwriting?
trinity: proper biped walking
increase range of mouse slider
gun shock based on damage source
-----------------------------------------
John Carmack's .plan for Sep 02, 1997
-----------------------------------------
* serveronly semeaphor close check
* error during initialization messagebox check
* software rendering default wall image
* unify slidestepmove
+ progress bar
+ roll angles are getting set on rotating models
+ blinking lights on bonus items
+ alpha test fix
bug with loading a new map after an errored out map
qe4: turn region off for new map
gun should show vertical acceleration on lifts
view angle turning based on impacts
muzzle flashes
trinity: fully compressed textures need to compress the mip levels as well, instead of generating them from the most detailed form.
trinity: different packing options to layout all the texture blocks. Square, thin, individual, etc.
trinity: investigate performance of background disk paging and clustering of texture blocks into disk allocation units
trinity: texture connectivity graph for prepaging?
trinity: speculatively upload things that might be seen next frame to balance uploading?
max upload, use lower mip levels if needed?
get rid of all the gl lightmaps in main memory!
allow jump up off bottom of water floor to give more velocity
merge net_udp and net_wins
slippery slopes
learned something: upload mip levels backwards
-----------------------------------------
John Carmack's .plan for Sep 03, 1997
-----------------------------------------
* fixed scalloc size 0
* box on planeside fix
* remove 0 and 255 colormap references
* don't allocate texinfos for empty texture names
* fix the initial teleport spawn timing bug
* sinking into plats
* exaggerate stepping when crouched?
+ allow pics off screen (status bar off bottom of screen)
+ check control configuration
+ make moveup jump
+ animating textures
+ base window not very noticable
+ any flowing?
+ fix deltaangle hack in putclientinserver
+ move copytooldorg to prepworldframe
+ demos
+ r_speeds include particles
+ escape should pause demos
reduce acceleration on low grav levels
moving translucent objects
timedemo leaves console in attract key mode
underwater speed is too great
flex legs intentionally on plats?
forward when facing an obstruction directly should not slide
clip brush fragments in base
exit button clip stuck problem
no such oldframe -1
melee attacks out of range
rename g_client to p_client
merge cl_fx and cl_tent
deal with oldorgs better
software menu black screen flickers
rethink scrap allocation
crouchwalk up slopes is fucked up (stair uping)
r_stats include bind counts
change lightmaps into images
make gl_bind() take an image, so it can reference sizes
script parsing should take /* */ and line continuation
reduce skies?
3dfx opengl: detect thrashing and split the cache?
demos don't read from pak files??????
muzzle flashes
forward / backwards airlocks
better button representations
brighter primary colors
-----------------------------------------
John Carmack's .plan for Sep 04, 1997
-----------------------------------------
* mcd alphatest workaround
* gl_finish
* gl_dynamic
* fixed crash without basedir
+ savegame harness
+ loading plaque
+ proper alias bbox in gl
+ light feedback for server
+ 3dfx vid_mode problem
+ remove MAX_SCRAPS
- arbitrary skin support needed for power armor
sort entities by texture
segment skies up more to get better caching?
circle monsters pvs
are gl sprites double bright?
-----------------------------------------
John Carmack's .plan for Sep 05, 1997
-----------------------------------------
* fixed all angle clamping issues
* allow look up / down to exactly 90 degrees
red numbers on status bar
-----------------------------------------
John Carmack's .plan for Sep 07, 1997
-----------------------------------------
* timedemo attract flag bug
* multiple model entities
* 8/16 bit updates
* merged delta with baseline
+ destroy windows on each ref start
+ server time clamping issues
+ client light level different between refs
+ remove version command
+ animating textures
+ run from cd option
+ toast mergedemos
flags not used in entity-state?
airborne frames for everything?
don't send player entity to owner in most cases
client quit dropping prints extra messages?
trinity: track and field style extra fast running?
-----------------------------------------
John Carmack's .plan for Sep 08, 1997
-----------------------------------------
* fixed time clamping issue
* removed win32isms from snd_mix.c and snd_dma.c
* optimized dsound locks
+ scale texture stats by texture size
+ not autodetecting PII for mono lightmaps?
+ put swaps from ref into qshared?
+ walk backwards when looking straight up?
seperate #define for asm code?
swim up with jump key
window close box
reload textures only on context recreation
do a stereo gl implementation on intergraph?
item using...
-----------------------------------------
John Carmack's .plan for Sep 09, 1997
-----------------------------------------
* freed textures on gl shutdown
* fixed pitch bounds check merge bug
* cleanup sound code
* cut default maxclients
* cut update_backup
+ soft fullscreen failure on ingr
+ error not closing window
+ cds window set on top of taskbar
+ fullscreen in mode 0, set mode 1 = crash
+ alt stuck down after alt-tab
fix vis expansion problem
trinity: shimmering heat atmospheric effects
need a remove command builtin for game logic
get all texture extension numbers into gl_image
vis decompression overrun
window doesn't offset in non-fullscreen modes
-----------------------------------------
John Carmack's .plan for Sep 10, 1997
-----------------------------------------
* don't precache player model in single player games
* dynamically change maxplayers
+ move null files into a seperate directory?
+ alt sticking
+ maxmoveleafs error
teleport in flash is still wrong on second level
stuck on wall with low grav jumping
allow minimize?
win95 memory paging
still have some tjunctions
-----------------------------------------
John Carmack's .plan for Sep 11, 1997
-----------------------------------------
* cddir
+ must save status upon entering a level if it was a new spawn
- each map has a unit number and a level number
is changing skill/etc going to be a problem while demos are running?
demos in pak file
don't use virtual alloc!
-----------------------------------------
John Carmack's .plan for Sep 12, 1997
-----------------------------------------
+ tag_game, tag_level
+ new game must clear
- need to save game state at last level entered as well as exact current
+ save level on exit..
+ spawnpoint to game dll wrong?
+ collapse sv_phys into something else?
+ skip all pixelformat stuff on minidrivers?
get rid of Com_SetServerState() ?
ERR_DROP?
status command that prints out server and client state?
don't allow anything but CRC checked pak file access in a demo
client userinfo change updates
worry about cvar info string lengths
make sure IP is visible to game
track deaths as well as fregs?
directmouse?
view shaking from explosions?
-----------------------------------------
John Carmack's .plan for Sep 13, 1997
-----------------------------------------
+ skill levels need to be archived with server state!
+ angle clamp on server is broke again
+ don't shrink status bar with window
+ make sure all char * in edicts are copies, not just references
+ difference between reentering a level and reloading it
check all savegame files for disk space errors.
current is automatically updated whenever a level is exited
archive the level being exited if not leaving the unit
save the map to be entered, SKIPPING ANY CINEMATICS!
end of game will not have a final map, so don't save
savegame does NOT update current, the level archive and server
is written directly to the new directory
new game
single player game
on death + press, bring up loadgame screen
deathmatch
on death + press, respawn
coop
on death + press, respawn
need to have the game start up without TCP/IP unless asked for
dir command with sys_find*
ping equalization?
set userinfo->ip on each server connect
high quality / low quality sound option in menu, create a special sample for
testing
fix svs / sv to be more game/level oriented
make coop games allways four player?
wav lag seems worse
-----------------------------------------
John Carmack's .plan for Sep 14, 1997
-----------------------------------------
* qdata grab alias numeric suffix
* menu architecture
+ make a portal entity
+ connect doors to portal entities
+ treat portal contents like windows
+ flood fill leafs, but stop at portals to count areas
+ if actual leaf with portal contents should chose any area next + to it
+ each portal brush should have exactly two areas bordering it
+ server sends over a bit vector of areas visible to player
+ use area visibility as fast reject for line testing?
+ should portal entities remain seperate, or just add a portal + field
+ to doors?
+ builtin: SetPortalState (int pnum, qboolean open);
+ portals MUST go in the structural bsp!
+ each leaf has an area field
+ each portal has two areas it connects
+ all other data can be derived
+ areas have the list of portals
+ area * area * 2^portals == too large!
+ must do dynamic flood fill
+ most portals will be closed, so flood fill is fast
+ game pause
+ no status bar after death
allow higher precision bmodel angle specification?
put v_idle back in?
super crouch jump?
establish a client connection at startup to avoid localconnects?
more bright areas in the game for contrast?
throbbing health status pic
weapon cycle command
bigger font?
allways have visible blood particles by face when hit?
bounce health status around when hit?
radius of alias model dlighting is greater than surface dlighting
trinity: use mouse curssors for ui stuff?
menu_move, menu_down, menu_up, menu_change, menu_slide
trinity: software trilinear with second pass? only works if vertex lighting
-----------------------------------------
John Carmack's .plan for Sep 15, 1997
-----------------------------------------
+ cinematic paking!
+ r_dspeeds should include translucent time
+ alt key stuck donw after alt-enter
+ bonus flashes
texture releasing from maps isn't uniqued
scissor triangles
faster z clip
make autoexec.cfg work differently because of demos
-----------------------------------------
John Carmack's .plan for Sep 16, 1997
-----------------------------------------
* finished box sweeping code
* fix the automenu key problem on bad cinematic
* blinking black screen palette set issues
* send dowarp over from server
* fixed color 0 grabbing
* bonus flashes
* Q_fabs
+ dedicated server
drop stair climb in air height
-----------------------------------------
John Carmack's .plan for Sep 17, 1997
-----------------------------------------
* fixed qe4 texturepath bug
* qe4: show in use textures even when showing a wad
* utils: fixed gamedir to allow nested quake2 directories
* moved env out of gfx, so gfx is all source files
malloc tags for game and level
clone detail brushes should remove detail flag
make timedemo a server connect thing
ktest.reg bad model
-----------------------------------------
John Carmack's .plan for Sep 18, 1997
-----------------------------------------
* finished code dump
* dedicated server
* removed all dash parms
* texture paging research
+ examine ambient sounds
+ key clear events doesn't clear everything
sound streaming
bsp hint brushes (SURF_NODRAW?)
autoexec.cfg
ip cvar for multihomed servers
ip userinfo for clients
report dash parms on cmd line?
menu on top of cinematic leaves crap
color 0 is still broken on NT
allow clients to connect to the server even if it is not
running a level?
local client is allways connected
clients are only kicked when the entire server is shut down
or they connect to a remote server
-----------------------------------------
John Carmack's .plan for Sep 19, 1997
-----------------------------------------
* basedir / cddir exec problem
* moved edict allocation into game
+ only change yaw on riding bmodels
+ city3 software crash
+ odd pause before connecting to map
+ !!!SV_PointContents needs to check entities!!!
+ areaportal numbers
+ move spawn/free into game logic?
are sound starts lagged by 0.1? only lag offsets?
get all cvars for game into one place
send objects only to owner or vs versa flag
loading plaque from post cinematic "you win!" screen
QE4: fix the idle redraw problem
vis decompress overrun
get rid of zombie state?
-----------------------------------------
John Carmack's .plan for Sep 20, 1997
-----------------------------------------
dead air conditioning
-----------------------------------------
John Carmack's .plan for Sep 21, 1997
-----------------------------------------
* areaportals!!!
* model contents for moving water
+ use different decompression buffers for pvs and phs?
+ fix headnode issues
get rid of fat pvs completely?
-----------------------------------------
John Carmack's .plan for Sep 22, 1997
-----------------------------------------
* fullsend headnode done properly
+ animating textures
+ must check all cluster areas for doors
-1 cluster issues?
more dramatic railgun spray
check all the trace flags to see if they are still needed
-----------------------------------------
John Carmack's .plan for Sep 23, 1997
-----------------------------------------
* fixed bmodel cluster overload
* double areas for doors
* drawworld 0
* bmodel sound origins
+ rename "globals" in game to ge?
+ remove limits on max entities in packet?
+ better way of tracking static sounds, so they can be turned off?
+ object looping sounds?
+ machine sounds in fact2
+ fix look up / run back bug
+ add more packet buffers to avoid "dumped packet" warnings?
+ dll basing information for function pointers
finish status bar and inventory
use areas for multicast
hint brushes
eliminate baselines by allways tracking client's knowledge of all ents?
qdata model frame auto-number issues
snap stuck view when dead?
set_ex command to set info status
game dll version number?
-----------------------------------------
John Carmack's .plan for Sep 24, 1997
-----------------------------------------
A significant new feature for map development sneaked into Quake 2 this week.
It has always been a problem with Quake that putting a door in front of a complex area didn't make the scene run any faster, unlike DOOM. In glquake, it actually made it significantly slower as you aproached the door, due to overdraw.
There was also the related problem that monsters heard sounds through doors even if they were closed.
This was because the primary culling mechanism used by Quake is the PVS - the Potentially Visible Set. It only knew about anything that you could POTENTIALLY see from your current (rough) position. If a door might open, the PVS would allways contain everything that you could see even if the door was currently closed.
Quake 2 now has a way to allow you to lop off large amounts of the map irrespective of the PVS.
A map is now divided into "areas" by "areaportal" entities, usually in door frames.
If the area behind a door is not reachable by any open areaportals, then nothing from that area will be visible or hearable. This helps both rendering speed and network bandwidth. It also give the level designer an easy "band-aid" when they have designed an area that is too slow.
Note that the area-reachable test is strictly a topological flood fill, so if there is ANY route to the other side of a door open, you will still be processing the area behind the door, even if there is no real way you could see through the available route.
If your level has a reasonable number of doors, it will often run at a fair speed without any PVS information at all.
To use this feature, you create a thin "func_areaportal" entity that hides completely inside the door, then target the door at it. Qbsp3 does a bunch of work behind your back that you really don't want to know about. Doors have special logic in the game to open or close the areaportal at the apropriate time.
I chose not to make it an automatic feature of doors for a few reasons:
1) Teamed double or quad doors would not create a single portal across the entire doorway.
2) The areaportal entity can also be used for things like exploding walls. You can even put one just around a corner and trigger it with a field, but it is usually better to just let the PVS take care of corner bends.
3) Complex doors would have created complex (but invisible) area portal brushes, which would have messed up the bsp a bit.
I think this was the very last data file change for quake II, so here is the current external files header for the curious: (4 character tabs)
[code]
//
// qfiles.h: quake file formats
// This file must be identical in the quake and utils directories
//
/*
========================================================================
.MD2 triangle model file format
========================================================================
*/
#define IDALIASHEADER (('2'<<24)+('P'<<16)+('D'<<8)+'I')
#define ALIAS_VERSION 8
#define MAX_TRIANGLES 4096
#define MAX_VERTS 2048
#define MAX_FRAMES 512
#define MAX_MD2SKINS 32
#define MAX_SKINNAME 64
typedef struct
{
short s;
short t;
} dstvert_t;
typedef struct
{
short index_xyz[3];
short index_st[3];
} dtriangle_t;
typedef struct
{
byte v[3]; // scaled byte to fit in frame mins/maxs
byte lightnormalindex;
} dtrivertx_t;
#define DTRIVERTX_V0 0
#define DTRIVERTX_V1 1
#define DTRIVERTX_V2 2
#define DTRIVERTX_LNI 3
#define DTRIVERTX_SIZE 4
typedef struct
{
float scale[3]; // multiply byte verts by this
float translate[3]; // then add this
char name[16]; // frame name from grabbing
dtrivertx_t verts[1]; // variable sized
} daliasframe_t;
// the glcmd format:
// a positive integer starts a tristrip command, followed by that many
// vertex structures.
// a negative integer starts a trifan command, followed by -x vertexes
// a zero indicates the end of the command list.
// a vertex consists of a floating point s, a floating point t,
// and an integer vertex index.
typedef struct
{
int ident;
int version;
int skinwidth;
int skinheight;
int framesize; // byte size of each frame
int num_skins;
int num_xyz;
int num_st; // greater than num_xyz for seams
int num_tris;
int num_glcmds; // dwords in strip/fan command list
int num_frames;
int ofs_skins; // each skin is a MAX_SKINNAME string
int ofs_st; // byte offset from start for stverts
int ofs_tris; // offset for dtriangles
int ofs_frames; // offset for first frame
int ofs_glcmds;
int ofs_end; // end of file
} dmdl_t;
/*
========================================================================
.SP2 sprite file format
========================================================================
*/
#define IDSPRITEHEADER (('2'<<24)+('S'<<16)+('D'<<8)+'I')
// little-endian "IDS2"
#define SPRITE_VERSION 2
typedef struct
{
int width, height;
int origin_x, origin_y; // raster coordinates inside pic
char name[MAX_SKINNAME]; // name of pcx file
} dsprframe_t;
typedef struct {
int ident;
int version;
int numframes;
dsprframe_t frames[1]; // variable sized
} dsprite_t;
/*
==============================================================================
.WAL texture file format
==============================================================================
*/
#define MIPLEVELS 4
typedef struct miptex_s
{
char name[32];
unsigned width, height;
unsigned offsets[MIPLEVELS]; // four mip maps stored
char animname[32]; // next frame in animation chain
int flags;
int contents;
int value;
} miptex_t;
/*
==============================================================================
.BSP file format
==============================================================================
*/
#define IDBSPHEADER (('P'<<24)+('S'<<16)+('B'<<8)+'I')
// little-endian "IBSP"
#define BSPVERSION 38
// upper design bounds
// leaffaces, leafbrushes, planes, and verts are still bounded by
// 16 bit short limits
#define MAX_MAP_MODELS 1024
#define MAX_MAP_BRUSHES 8192
#define MAX_MAP_ENTITIES 2048
#define MAX_MAP_ENTSTRING 0x20000
#define MAX_MAP_TEXINFO 8192
#define MAX_MAP_AREAS 256
#define MAX_MAP_AREAPORTALS 1024
#define MAX_MAP_PLANES 65536
#define MAX_MAP_NODES 65536
#define MAX_MAP_BRUSHSIDES 65536
#define MAX_MAP_LEAFS 65536
#define MAX_MAP_VERTS 65536
#define MAX_MAP_FACES 65536
#define MAX_MAP_LEAFFACES 65536
#define MAX_MAP_LEAFBRUSHES 65536
#define MAX_MAP_PORTALS 65536
#define MAX_MAP_EDGES 128000
#define MAX_MAP_SURFEDGES 256000
#define MAX_MAP_LIGHTING 0x200000
#define MAX_MAP_VISIBILITY 0x100000
// key / value pair sizes
#define MAX_KEY 32
#define MAX_VALUE 1024
//=============================================================================
typedef struct
{
int fileofs, filelen;
} lump_t;
#define LUMP_ENTITIES 0
#define LUMP_PLANES 1
#define LUMP_VERTEXES 2
#define LUMP_VISIBILITY 3
#define LUMP_NODES 4
#define LUMP_TEXINFO 5
#define LUMP_FACES 6
#define LUMP_LIGHTING 7
#define LUMP_LEAFS 8
#define LUMP_LEAFFACES 9
#define LUMP_LEAFBRUSHES 10
#define LUMP_EDGES 11
#define LUMP_SURFEDGES 12
#define LUMP_MODELS 13
#define LUMP_BRUSHES 14
#define LUMP_BRUSHSIDES 15
#define LUMP_POP 16
#define LUMP_AREAS 17
#define LUMP_AREAPORTALS 18
#define HEADER_LUMPS 19
typedef struct
{
int ident;
int version;
lump_t lumps[HEADER_LUMPS];
} dheader_t;
typedef struct
{
float mins[3], maxs[3];
float origin[3]; // for sounds or lights
int headnode;
int firstface, numfaces; // submodels just draw faces
// without walking the bsp tree
} dmodel_t;
typedef struct
{
float point[3];
} dvertex_t;
// 0-2 are axial planes
#define PLANE_X 0
#define PLANE_Y 1
#define PLANE_Z 2
// 3-5 are non-axial planes snapped to the nearest
#define PLANE_ANYX 3
#define PLANE_ANYY 4
#define PLANE_ANYZ 5
// planes (x&~1) and (x&~1)+1 are allways opposites
typedef struct
{
float normal[3];
float dist;
int type; // PLANE_X - PLANE_ANYZ ?remove? trivial to regenerate
} dplane_t;
// contents flags are seperate bits
// a given brush can contribute multiple content bits
// multiple brushes can be in a single leaf
// lower bits are stronger, and will eat weaker brushes completely
#define CONTENTS_SOLID 1 // an eye is never valid in a solid
#define CONTENTS_WINDOW 2 // translucent, but not watery
#define CONTENTS_AUX 4
#define CONTENTS_LAVA 8
#define CONTENTS_SLIME 16
#define CONTENTS_WATER 32
#define CONTENTS_MIST 64
#define LAST_VISIBLE_CONTENTS 64
// remaining contents are non-visible, and don't eat brushes
#define CONTENTS_AREAPORTAL 0x8000
#define CONTENTS_PLAYERCLIP 0x10000
#define CONTENTS_MONSTERCLIP 0x20000
// currents can be added to any other contents, and may be mixed
#define CONTENTS_CURRENT_0 0x40000
#define CONTENTS_CURRENT_90 0x80000
#define CONTENTS_CURRENT_180 0x100000
#define CONTENTS_CURRENT_270 0x200000
#define CONTENTS_CURRENT_UP 0x400000
#define CONTENTS_CURRENT_DOWN 0x800000
#define CONTENTS_ORIGIN 0x1000000 // removed before bsping an entity
#define CONTENTS_MONSTER 0x2000000 // should never be on a brush, only in game
#define CONTENTS_DEADMONSTER 0x4000000
#define CONTENTS_DETAIL 0x8000000 // brushes to be added after vis leafs
#define CONTENTS_TRANSLUCENT 0x10000000 // auto set if any surface has trans
#define CONTENTS_LADDER 0x20000000
typedef struct
{
int planenum;
int children[2]; // negative numbers are -(leafs+1), not nodes
short mins[3]; // for frustom culling
short maxs[3];
unsigned short firstface;
unsigned short numfaces; // counting both sides
} dnode_t;
typedef struct texinfo_s
{
float vecs[2][4]; // [s/t][xyz offset]
int flags; // miptex flags + overrides
int value; // light emission, etc
char texture[32]; // texture name (textures/*.wal)
int nexttexinfo; // for animations, -1 = end of chain
} texinfo_t;
#define SURF_LIGHT 0x1 // value will hold the light strength
#define SURF_SLICK 0x2 // effects game physics
#define SURF_SKY 0x4 // don't draw, but add to skybox
#define SURF_WARP 0x8 // turbulent water warp
#define SURF_TRANS33 0x10
#define SURF_TRANS66 0x20
#define SURF_FLOWING 0x40 // scroll towards angle
#define SURF_NODRAW 0x80 // don't bother referencing the texture
// note that edge 0 is never used, because negative edge nums are used for
// counterclockwise use of the edge in a face
typedef struct
{
unsigned short v[2]; // vertex numbers
} dedge_t;
#define MAXLIGHTMAPS 4
typedef struct
{
unsigned short planenum;
short side;
int firstedge; // we must support > 64k edges
short numedges;
short texinfo;
// lighting info
byte styles[MAXLIGHTMAPS];
int lightofs; // start of [numstyles*surfsize] samples
} dface_t;
typedef struct
{
int contents; // OR of all brushes (not needed?)
short cluster;
short area;
short mins[3]; // for frustum culling
short maxs[3];
unsigned short firstleafface;
unsigned short numleaffaces;
unsigned short firstleafbrush;
unsigned short numleafbrushes;
} dleaf_t;
typedef struct
{
unsigned short planenum; // facing out of the leaf
short texinfo;
} dbrushside_t;
typedef struct
{
int firstside;
int numsides;
int contents;
} dbrush_t;
#define ANGLE_UP -1
#define ANGLE_DOWN -2
// the visibility lump consists of a header with a count, then
// byte offsets for the PVS and PHS of each cluster, then the raw
// compressed bit vectors
#define DVIS_PVS 0
#define DVIS_PHS 1
typedef struct
{
int numclusters;
int bitofs[8][2]; // bitofs[numclusters][2]
} dvis_t;
// each area has a list of portals that lead into other areas
// when portals are closed, other areas may not be visible or
// hearable even if the vis info says that it should be
typedef struct
{
int portalnum;
int otherarea;
} dareaportal_t;
typedef struct
{
int numareaportals;
int firstareaportal;
} darea_t;
[/code]
* fixed area bug for headnoded entities
* fixed noclip outside world view with areas
* fixed gl_lienar getting stuck
+ cull sound spatialization by area?
+ don't save level if going to a new map instead of gamemap
+ put .pak format into qfiles
- make chaingun do less damage per bullet than the machinegun?
+ check entity sound overriding in fact2
+ software glitches with areaportals
+ move spawn and free into game
+ weird blaster trails
+ make sure doors open / close areas properly reletive to sound starts
guarantee string fields are never NULL?
linksequence
client userinfo updates
software skybox rotation
unlinkentity?
make -ip work as a cvar
ip as userinfo
areaconnected game primitive
server engine manages
communication
connection establishment
collision detection
console interface
map/game transitions
cache pvs/phs expansion
falling damage
manual mipmapping on skins?
-----------------------------------------
John Carmack's .plan for Sep 25, 1997
-----------------------------------------
+ should loadgame allways unload and re-init game dll?
+ load level with spawnpoint is different than load game
+ good sound control instead of staic sounds
+ move use / drop into game code
+ texpaint autosave
+ noreadlevel cvar?
localconnect sometimes needs to be reissued
non-axial triggers
rename and g_client to p_*?
don't nextlevel from "you win"
userinfo visible to progs
too many edicts in an area for sv_world?
spawn flashes are still wrong
-----------------------------------------
John Carmack's .plan for Sep 28, 1997
-----------------------------------------
+ pain sounds?
window crunching on win95, due to order of DX operation?
-----------------------------------------
John Carmack's .plan for Sep 29, 1997
-----------------------------------------
* texpaint: backup files
* texpaint: size dialog on new skin
* texpaint: auto save
* normailized translucency level in gl
* fixed func_group entity miscount
* fixed target_changelevel use clearing
* fixed pointcontents with moving entities
* fixed spawnpoint storage
* use areas in multicast
* removed ambient sound calls
+ >512 entities
+ view kick even without move kick
+ blaster spawn point
- include sky, skyaxis as player_state variables?
+ transfer player health on each spawn and level exit
set spawntemp strings to "" instead of null?
alias save sequence number issues in qdata and texpaint
client userinfo
different blaster flash against flesh
rename edict_t to entity_t
init without IP
never same pain animation twice in a row?
map during loading bug
remove old_origin
makeuserinfo / makeserverinfo?
makeclientcommand?
auto-loop entitystate sounds?
!!! must include full path of predicted positions in fatpvs !!!
muzzle flash effect in entity_state?
jail4a iris door problem
MOTD console variable
QE4: scrub out partial detail brushes
-----------------------------------------
John Carmack's .plan for Sep 30, 1997
-----------------------------------------
* better host_speeds
* fixed bsp slowdown bug
* made Draw_Pic: bad coords not an error
* avoided double game init on loadgame
* moved serverflags into game
* fixed pause on initial connect from dumped loopback
* inventory rate bug
* client view rolling bug on level change
* RF_GLOW
* texture animation
+ more barrel explosion damage
+ archive all sensitive server cvars
+ fix inventory system
+ target_goalcounter
+ get rid of packet_entities_t as a structure
+ cinematic message name bug
+ up as jump?
+ more light!
+ weapons need to be more obvious in the world. light pulsing?
+ make first backtrack into an area with known landmarks
fade center printed text
still have color 0 problems under NT
allow game to select a client slot for connections?
combine g_player with something else
save view angles in savegame somehow
more function pointer checks in loadgame
watch out for different maxclients on loadgames
increase alias model shading level?
use userinfo_sequence
HUD strings need to be tag_game, not level
make ping time available to game?
different faces on status bar for male/female characters?
save health across levels
stuck in water currents in base2?
clamp max fly velocity
fish in ait on base3
guys shooting through force field sometimes in bunk
rename gl_mesh to gl_alias? or r_alias to r_mesh?
jump / crouch key placement
auto view centering
more edge on mipmapping in software?
-----------------------------------------
John Carmack's .plan for Oct 01, 1997
-----------------------------------------
* EF_ANIM
* got rid of precache_
* got rid of SV_Error
* !!! config strings !!!
+ are lightstyle strings being dynamically freed properly?
+ pause
+ remove SV_Error?
!!!move timedemo to server
should setmodel take an index?
smart precache of weapons?
long crawls are annoying
skin reference counting
does leak test work?
bad surface extents levels
make sound and image names include extensions?
!!! how to download implicit images ??? !!!
!!! demo recording with deltas needs to wait for full update !!!
make timeout at least a minute?
multicast_all_r for configstring should go to connected as well as active
clients
string encode SKY_AXIS and SKY_ROTATE in SKY?
it will be possible to get an index for an item not yet known
because of reliable / unreliable issues
block_until_reliable option?
supress flag on HUDs to allow cheap blinking?
rename "map" to "start"?
extra packet dumps still happening on map start
remove CL_MapEntity
move baselines into a parallel array?
don't expose svc_tent / muzzle flash numbers to game?
dropcommand cvar to restart crashed servers?
better box top walk jumping
full death cycle for player
inventory is persistant, per-client state.
no high step jump out
pain and death animations should be based on impact direction and
total damage in that frame
check on virtual alloc / commit issues
weird bmodel edge stream problem
increase numstacksurfs / numstackedges
clear sound buffer on loading plaque
-----------------------------------------
John Carmack's .plan for Oct 02, 1997
-----------------------------------------
* don't lerp blends
* sum damages for end of frame
* damage kicks scaled by health
* don't run more than one frame at a time
* fixed alias model brightness in software
* equalized light feedback value between refs
different console background
infantry melee attack?
still get stuck sometimes
mono lighting should not color alias models
put away restart game menu
rename qmenu and menu.c to something common
numeric keypad controls?
monster hearing not right?
walk into player = allways attack
are infantry fullbright flashes not working?
level to level health
lighting feedback still different in software and gl
player pain sounds
animate translucent textures
better monster sight / hearing
make sure switches are animating
echo center prints to console
PHS or PVS activated guards?
rotate with textures option in QE4 for crates
do mynoise entities leak on level transitions?
crouch strafe is still full speed
-----------------------------------------
John Carmack's .plan for Oct 03, 1997
-----------------------------------------
* game pause
* pain sounds
* save health between levels
* moved baselines to a parallel array
* software screenshot directory
* 1.4k packets only!
* map command while paused?
* pos sound overriding
* sound area testing
* seperate pvs / phs static arrays
* cleared sound buffer when disabled for loading
* PHS calculation bug
sound improvements since q1
respatializing on moving entities
sub frame start commands
looped sounds are deterministic
sounds are removed by area and PHS
looped sounds sum
muzzle flashes!
bullet impact puffa?
why can't you fire a single machinegun bullet?
avoid loading the map file twice for server and refresh
option for multicast to PVS for effects instead of PHS?
remove sbar2 sounds
centralize all communication between client and server sides
warnings for improperly looped sounds
multiple speaker entities with the same looped sound
will just increase the range
target_speaker checkbox for player locals (voiceovers)
base1 - base2 - base1 - base2 -death goes to base1
paused level to level bugs
yaw towards killer
monsters not going to ideal yaw when shooting?
blood jet muzzle flashes
iventory update on level change
keepalive messages while precaching
no red flash on deaths?
initial and final trail parts
server quit doesn't get the disconnect message out
option to have dlights backface cull
extended sound bytes
release mouse when paused?
nver let server be connected without local client
!!! possound needs to also take an entity number !!!
slow water wading
-----------------------------------------
John Carmack's .plan for Oct 04, 1997
-----------------------------------------
* map_noareas
* target_speaker
handle bmodel origins on client side, search for good area
check localsound pos starts
make the server read the demo configstrings and baselines
and spit back to the client like normal
water wading sounds
sound streaming option
remove cl_mapentity
sync camera pain motion to sound length?
give all sounds a max volume area isntead of instant diminish
quakeworld style shotgun handling
-----------------------------------------
John Carmack's .plan for Oct 05, 1997
-----------------------------------------
Developing for windows is not fun. We are having a lot of trouble getting good solid compatability across all the systems we are testing on.
When it works right, it just pops right into full screen mode with sound and network just like the dos version, but we are still chasing problems on several systems. Sigh.
* !!! autolooped entity sounds !!!
make all tools into 5.0 projects
combine SZ_ and MSG_
allways mkdir gamedir?
pause dumps packets?
clear all background all the time flag
player physics
MD4 each map file?
print version number on console bottom
select a different cd track if all goals accomplished
get rid of alphalight
-----------------------------------------
John Carmack's .plan for Oct 06, 1997
-----------------------------------------
* larger bsp token length
goal sound, secret sound, help sound
remove pushmatrix/popmatrix
less shademodels
sound mixaheads
flies should be a sound field
trinity: two pass texture checking to avoid thrashing?
lose links and $ macro expansio?
lose Com_SetServerState
blinking flags on huds (blink F1)
!!! rotating object view changes not in yaw !!!
save configstrings in level to get lightstyles
better armor feedback
armor sounnd?
-----------------------------------------
John Carmack's .plan for Oct 07, 1997
-----------------------------------------
* pitch snap clamping
* clamp at 89
* kill sounds when loading plaque
* no fov or null pic problem during prep refresh
* wounded faces
die then bring down console over menu bug
death
windows key
make sprite files text format?
all explodables and breakables should be pre-broken in dm
flash stats on change?
send email to j sturges
inventory names
super tank skin
variable delay on centerprint
timedemo
demo tests
flies as entity sound
release mouse when paused?
centerview
peak to peak view bobbing
counter items
infantry skins
menu sounds
secret sound
goal sound
sound when low on health?
respawn muzzle flash event still wrong
falling damage
rotating sky in software
color 0 on NT
transparent water insides
-----------------------------------------
John Carmack's .plan for Oct 08, 1997
-----------------------------------------
* fixed entity numbers 512-1024
* combined baselines and oldorgs
* demos working again
* MAX_SFX bug
spawn invisible allways starts at 256, so visible get bytes?
check goal counting
any key puts away help?
muzzle flashes
make cl_entities dynamic?
removed mergedemo on client
remove all client demo playback
finish savegame / loadgame UI
finish cinematic sound
check demo fopen spawning for cddir
weird palette issues?
more red
minimum health
infantry muzzle sounds?
remove blaster hit flashes on flesh
no savegame when dead
mine2 dissapearing problems
use key problems
qrad: infinite styles on face
allways have two secrets
allways select new items
do demos need a precache command?
timegraph
!!!save lightstyles in savegame!!!
!!!save areaportal state in savegame!!!
are loadgames doing 10 second prerun?
move say and say_team into game
sentity_t gentity_t
more blood
areaportals in software - bad sort keys?
-----------------------------------------
John Carmack's .plan for Oct 09, 1997
-----------------------------------------
* removed MAX_PACKET_ENTITIES limit
* used areas for beam culling
* centerprint to non client not error
* don't rotate roll when pushed by entities
* areaportal fragments in software
* F_CLIENT fix
* KEY_ANY fix
* save areaportals
* save lightstyles
* fixed secret double counting
* up as jump
color 0 on NT
water wading speed
water jump out
centerview
no savegame when dead
mroe damage blend
putclient in server shouldn't reference weaponmodel
userinfo issues
IP cvar for servers
IP userinfo for clients
remove sv.viewpos?
make max_entitites a noset cvar
don't use PHS?
up / down issues
broadcast centerprint
intermissions
flickery lights
free mouse when paused
-----------------------------------------
John Carmack's .plan for Oct 17, 1997
-----------------------------------------
Somehow we managed to convince the mesquite city services and police department to let us take our cars down to the municipal airport and run them down the runway to get radar speed numbers. Is that cool, or what?
So, how fast can we go on a 6000 ft runway and still stop before running off the end?
John Cash's M3 just barely hit the 135 mph speed governer.
Bear's turbo supro hit 144
My F40 hit 165
My TR's left turbo exploded at 160 mph :(
Adrian, Todd and Paul couldn't make it, so we didn't get viper, vette or porsche numbers.
It took less than 2000 ft for the TR to do 160. We were fully expecting to do 200 mph in 4000 ft if things had held together.
We have a bunch of video and sound footage that we are going to digitize eventually. We made one run with a police mustang chasing after my F40. Guess who won.
The F40 is a very, very durable car. I made six runs around 160 mph, and it didn't even fade. Same thing on a racetrack. Lap after lap without any changes. My TR makes 1100 hp for twenty seconds, then explodes..
-----------------------------------------
John Carmack's .plan for Oct 19, 1997
-----------------------------------------
I hope everyone is enjoying the quake 2 test.
Its always hard to release a version of a product that you know isn't in its final form. There are plenty of things that are getting better every single day, but we need to chop it off at some point to let everyone test it out.
We will do another demo after we finish the full retail product, so if you don't like looking at preproduction stuff, wait for that one.
Still, I am pretty happy with the test. I think Quake 2 is definately the most cohesive game we have ever done.
Don't worry - just because the test doesn't have multiplayer in it, it doesn't mean that we haven't been thinking about it. Many features in the quake 2 architecture are going to enable a whole new level of net play. It will take a few months after the full release for all the potential to start showing through, but just you wait!
The biggest changes to Quake 2 are internal. Anyone doing modification work on Quake is going to be ecstatic when they get to work with quake2. The game dll source code and all the utilities (including the OpenGl map editor) will be released shortly after the game hits store shelves.
-----------------------------------------
John Carmack's .plan for Oct 20, 1997
-----------------------------------------
Many of the comments about the Quake 2 test are already being addressed. We expected quite a few of them, but the test has served its purpose of bringing in some good feedback that we couldn't have predicted.
The final game will definately be better as a result of the test.
However, it certainly won't please everyone. I am confident that the majority will think that Quake 2 is significantly better than anything we have ever done before, but even if we please 80% of our potential customers, that will still leave a couple hundred thousand people thinking that we let them down.
I suppose that I have it the easiest there - I can always defend my technical decisions with specific discussions of my evaluations of the tradeofs that led me to the paths I chose. In fact, in a large number of cases when someone suggests something, I can actually say "Tried it. Didn't work as well."
Defending level design, artwork, or sounds is a lot harder. We can't even always agree here at id on many of these issues, so we know for sure that we can't please all the users simultaniously. All we can do is put talented people on the job and have confidence in their abilities.
Note: Q2TEST DOES NOT INCLUDE ANY HIGH QUALITY SOUNDS! That would have added another 15 megs to the demo size. Selecting high quality sounds just upsamples the existing 11khz / 8 bit sounds. There is a significant quality increase (at a slight speed and memory cost) with the full production sounds.
Quake 2's goal is to be the best first person shooter ever. We are trying to evolve a genre, not move to a different one. If you don't want a game that mostly consists of running around and killing things, you will be dissapointed. We are trying to be cohesive, but not deep. I have high hopes for the games that are atempting to aply our technology to other genres, but don't look for it in Quake 2.
A quick plug:
if you have any interest in programming, you should look at Michael Abrash's Graphics Programming Black Book Special Edition. It has just about everything he has written, from the ancient work on optimizing for the 8086 (still interesting to read) to the articles written during quake's development.
I personally learned a lot from Michael's early articles, and I was proud to contribute to the later ones.
http://www.amazon.com/exec/obidos/ISBN=1576101746/3226-6070174-877180
-----------------------------------------
John Carmack's .plan for Nov 01, 1997
-----------------------------------------
* interpolate prediction error
* fixed farthest respawn
* removed backspeed
* no pickup weapons when dead!
* multiple crosshair pics
* fixed dropping items in wall
* disabled auto weapon switching in deathmatch
* respawn_time
* mroe precaches
* removed doubles
+ pitch around bug
+ ping calculation
+ are demos broken with prediction?
+ no footsteps if moving slow?
+ kill self command
+ no toss weapons in wall
+ footstep doubletap
+ clear gib flag on respawn
+ faster weapon deop times
precache talk wav
qbsp: MAX_MAP_AREAS when leaking?
flies effect on hyperblaster???
no slide under staircase
make blaster bolt move faster
BECOME_FLASH effect?
make dedicated server sleep
connect to other server while playing bug
bit code net messages?
win95 joining
changeweapon work better when out of ammo
pop in maps?
highlighted numbers
sort image_t lists?
switching rules
brain effect
monsters source shots before testing line of sight
secret doors
obituaries from monsters
warp gate effect
teleporter effect
increase max switched lights
max lightstyles bug
shorter wav latency
beep beep on pc icon
blood jets from pain skins
gibs shoot up out of lava
savegame in water bug
font outlines not sharp in gl
weapons vanishing on toss?
longer pause after death before respawn
temp invulnerability after respawn?
telefrag not always working?
step up in water?
clear powerups on death
don't hold a grenade when none left
l_health item precache?
-----------------------------------------
John Carmack's .plan for Nov 02, 1997
-----------------------------------------
* fixed pitch clamping
* Com_PageInMemory
* fixed menu cursor time
* net connect when playing bug
* custom skins
* fixed server update without game update bug
+ bump version numbers
+ don't go to half console until connected
+ delay before firing bfg
+ stairup allows wall climbing now
+ previous frame issues
+ don't copy all of frame.packetentities
no server pause in dm
normalize skin texture coords for software
splashing sound when swimming in water
ipx
software underwater surfaces
nopredict option at server
scroll inventory
smooth step up
remove rand1k
fire func_explosive targets when starting in deathmatch
-----------------------------------------
John Carmack's .plan for Nov 03, 1997
-----------------------------------------
* fixed divide by zero in kickback
* fixed overflow
* fixed walkmap up slopes
* bumped versions
* grabbed all cinematics
* fixed abort intro issues
* kill command
* fixed server status command
* more weapon precaches
* noexit by default
* autoremove some stuff in deathmatch
* make game initialization like - sound -
* pumped message loop during caching
* client persistant data
+ no drop weapons without ammo
+ no footsteps when walking
+ saved across kills
+ saved across levels in single player
+ client levelstate
+ clear client times on level change
+ shouldn't be dumping unreliable messages
+ palette changes on cinematic
laod game should throw loading plaque immediately
not pausing when menu is up!
"don't need" sound for no pickup?
min_intermission cvar?
run key should be a toggle
dropped items respawn
auto use items
powerup sounds
print sound precache pacifiers
sendkeyevents during loading?
better pingservers
alias models are lit outside ofdlight ball
sv_shownet
"killed by" icon on scores?
don't allow dedicated without deathmatch
map transitions, but gamemap doesn't?
intermission spots with deathmatch
blinking health indicator
blinking computer indicator
move swapbuffers and add flush to glquake?
chaingun sounds off on NT?
test ping on serial port to serial port connection on win95
console prog commands
lower the scoreboard readout so you can read the obituary bessage
-----------------------------------------
John Carmack's .plan for Nov 04, 1997
-----------------------------------------
* allowed bad sky textures
* cinematic tweaks in gl
* high res skins
* fixed duck speed issues
+ names are messed up
+ not disconnecting cleanly?
+ cinematic GL wrapping problem
+ cinematic quality issues
+ cinematic sound
+ cinematic end frame marker
+ switch sound to high quality for cinematic
+ is idlog aborting early on menu?
+ dropping items makes them respawn in deathmatch
+ crouch strafe is faster than forward
+ sink in plats bug
+ lower paused icon on screen
+ min firing ammo for dropping weapons
+ spawn explosions with a random yaw
gun puff animations wrong?
no status bar during intermission
sv_showcmds
are sky images freed properly?
wading sound
use 16 to 8 table is ref_soft for tga loading?
savegame off pak file
demo file parsing from pak?
check replace alpha value for mcd hack
screen update timer for software opengl?
instant items
item sounds
include texture source size in texinfo so other scaled versions can be made?
are cinematics using color 0?
send pak checksum to server?
fix dedicated_start
print dm rules on connect?
blink f1 and play sound
skill values!
loadgame from console
input based demos for profiling
-----------------------------------------
John Carmack's .plan for Nov 05, 1997
-----------------------------------------
* fixed sink into plat bug
* fixed scoreboard display between deathmatch levels
* seperated game dll definitions
+ clamp low cin times
clear angles on loadgame
check client entering during intermission
seperate headers for monsters and players
-----------------------------------------
John Carmack's .plan for Nov 06, 1997
-----------------------------------------
* s_testsound 1
* fixed streaming sound on 95
* streaming sound at full volume
* removed multiply from mixing
* khz change for cinematics
* blaster precaches
* fixed cinematic from pak streaming
* don't use primary sound buffer option
+ precache blaster
+ dropped grenades on death shouldn't respawn there..
set hostname by ip hostname
timegraph not right
rename map to start
check entire game without asm code
no mouse cursor when fullscreen
die with grenades needs to stop ticking sound
high quality sound directories
-----------------------------------------
John Carmack's .plan for Nov 07, 1997
-----------------------------------------
* flag reorg
* teleporters
* put holdangles into pmove.pm_type
+ pm.touchents holds duplicates
+ damage anything flag
+ precache chat sound
+ teleporters at player spawn points
+ remove rocket fragments in dm
rename entity_t to rentity_t ?
teleport sequence bit to make ef_teleport reliable?
turn any event into a temp entity? (with or without angles)
unify sound starting as temp entities?
is time being over quantized by timegettime?
order events by priority
login / logout as events?
all sound channels as extra events?
trinity: objects should have enabler inputs as well as multiple impulse targets
-----------------------------------------
John Carmack's .plan for Nov 08, 1997
-----------------------------------------
* make random respawn option default and work
* don't drop empty weapons
* teleport angles
* teleporters at player spawn points
* fixed telefrag self on respawn
* fixed userinfo on initial entering
* precache land sounds
* don't change console height until connection packet
* a disconnecting client shouldn't generate a badread
* remove rocket fragments in dm
* damage anything flag
* don't call duplicated pm.touchents
* client parse entities array
* no weapon toss in single player
sound dies after several hours?
scoreboard faces
show killed by face on scoreboard
deal with old_origin properly
delay cdtrack play until connected
can't escape out of loadgame menu when dead?
-----------------------------------------
John Carmack's .plan for Nov 09, 1997
-----------------------------------------
* no client pmove at all with prediction off
* railgun crashes
* fixed rub stuck bug
+ gravity in pmove
+ control config is messed up
+ seperate client event processing from parsing
+ are baselines not working right?
make-item-selected command
notched look up / down commands
keyboard look
map name is messed up on start server
deathmatch character weapons
unify uses of ent / client / etc in source?
no player actions when paused
check incoming_acknowledged for pings
master servers
dm respawn isn't clearing pitch?
teamed teleport destinations
teleport angles
# form for kicking players
make showinventory a stat bit, like layout
unify all data file references for easy downloading?
how to handle different quality versions?
if crushed in dm, respawned outszide world?
shadows
rename entity_t->flags to renderfx
checksum client to server messages
smooth out step up
does spectator work?
trinity: coarse radiosity source lattice for dynamic lights?
oldorigin issues
replace with a previous state send?
-----------------------------------------
John Carmack's .plan for Nov 10, 1997
-----------------------------------------
* qe4 bad class parse bug
* grenade bounce generates two sounds
* playerstate delta
* qdata variable sound rates
check the replace mode alpha bug
replace all muzzle flashes with events
lower railgun
-----------------------------------------
John Carmack's .plan for Nov 11, 1997
-----------------------------------------
* cinematic playback at variable sound rates
* qdata multiple video in single file fix
* only one token huffman bug
* demowaiting
* allowed individual reliable overflows
* make all clients invisible at intermission point
* mask high bits in client_t->name
* full delta compression
don't show paused plaque in dm
check all serverinfo flags (skill, nomonsters)
crunch the scoreboard data a lot
dedicated net thread
random-not-nearest option
"public" variable
check the "using previous_origin" notes
echo chats to console?
weapon icon when hand is centered
game skills
better console keyboard editing
are demos broken with current delta compression?
should client string commands be ties to usercmd_t, to fix drops?
why is ref_gl.dll as large as it is?
allow round up and >256 textures on gl
mouse during cin
echo chats to console
is the dedicated server sleeping?
get mins/maxs from pmove
warn at startup if any spawn point is in a wall
make teleport pads glow?
never make water solid for entity cull purposes?
get sound position needs to check the valid count on entities
make secondary sound buffers the default?
pak file sorting by traces?
no colored lighting with mono lightmaps!!!
option to make picking up items not select
change stretch-raw to a dedicated full screen blit for better performance?
bsp to do:
=========
robustness
water problems
increase MAX_MAP_ENTSTRNG
allow any number of light styles
MAX_MAP_AREAS when leaking?
-----------------------------------------
John Carmack's .plan for Dec 01, 1997
-----------------------------------------
Quake 2 has mastered.
Where we go from here:
Point release.
We should have a Quake 2 point release out shortly after the game gets in your hands. We intend to fix any bugs that turn up, improve the speed somewhat, and optimize for internet play in various ways. We will also be making several deathmatch only maps.
Deathmatch in Q2 has gotten a lot of lan testing (Thresh, Redwood, and Vik Long helped quite a bit the last week with tuning), but not much internet testing. There are probably gaping holes in it, but we will address them soon.
The deathmatch code in the shipping Q2 is also not designed to hold up against malicious users - there is no protection against clients being obnoxious and constantly changing skins, chat flooding, client-side cheating, or whatever.
Q2 does checksum maps on the client side right now, so cheater maps won't work like they do in Q1, but cheater models and skins are still possible. I have some plans to combat that in the point release, but there are a lot of forms of cheating that can be implemented in proxies that are fundamentally not detectable. I can make it very painfully difficult for people to implement such things, but a very clever person with a dissasembler just can't be stopped completely.
The server code and network protocol should be able to support ultra-large player counts, but I know I need to do some low-level work to get around operating system buffer limits before it will actually work. We will test at least a hundred players in a giant map for the point release, but we won't actually address the issues of making a rational game at that level (chat hierarchies, team spawning, etc). I am very much looking forward to seeing what the user community creates on that foundation.
It is likely that the point release may have incompatable network protocols and savegames. Fair warning.
Q2 Demo.
After the point release, we will be making a new demo release. If you experienced compatability problems with q2test, or were unsatisfied with the quality in some way, you should look at the demo. The final product is much improved.
Q2 Ports.
We are commited to Win32 Alpha, Linux, irix, and rhapsody in that order. It is likely that a bunch of other ports will come later, but no promises. The presence of hardware-accelerated OpenGL on a platform will improve it's odds a lot. Zoid will probably prioritize Q2 CTF over other ports, so hold off on bugging him about ports for a while.
Development tool release.
I will basically be making publicly available a subset of the directory tree that we will deliver to our licensees. All the utility source code, the game dll source code, and probably some example source media - .map files, artwork, model source, etc.
Q2 mission pack.
Most of the company will be working on a mission pack while Brian and I write tools and technology for trinity.
Trinity.
I am going to rapidly wean myself off of working with quake so I can concentrate fully on new directions. The evolution of the Q2 codebase will be left to John Cash (until the mission pack ships) and Zoid.
Everyone should keep in mind that any next-generation game that we produce is a LONG way off, so don't start getting all worked up over it, ok?
For the curious, it does look like java is going to start playing a significant role in our future projects. All of the lightweight utilities will be java applications (some requiring OpenGL bindings). The heavy duty number crunching utilities will probably stay in C. It is still unclear how much of the game framework and the level editor we can get away with doing in java.
-----------------------------------------
John Carmack's .plan for Dec 02, 1997
-----------------------------------------
A couple things I forgot to mention:
DOOM source. Still planned to be released Real Soon Now, but there is some work that needs to be done on it first to remove the sound engine, which was written by someone else.
Our Quake editor. It will be released with the tools, but it really isn't going to be all that usefull to many people. Most people will be better off with one of the actively supported editors designed for normal machines.
There is no documentation (Steve Tietze at Rogue has talked about writing something, though). It is designed to run at 1280*1024 resolution on a fast, fully-compliant OpenGL driver. It was designed for high-end boards like intergraph realizm, 3DPro, and Glint boards, but it also runs ok on 8 mb consumer boards like the permedia II and rendition V2200. It will NOT work with voodoo or powerVR. It is unlikely to work with voodoo rush, because of framebuffer size limits, but it might work at a low screen resolution. It might be workable on RIVA cards if they do some fancy work disposing buffers between window renderings (they are a 4mb card, but the textures can stay in AGP memory, so it will almost be enough). I'll work with them if they want to give it a try.
Right now, only 3Dlabs has a full opengl driver on win-95 (and it is a little flaky). All the other cards would require you to run NT. Over the next several months, most of the major vendors should be releasing full OpenGL drivers that work in '95, but there are no firm release dates.
A comment to the people complaining about the release not having Their-Favorite-Feature:
A software project is never, ever completely finished. If you wait until EVERYTHING is done, you won't ship at all.
Would it have been the right thing to delay releasing Quake 1 until I had written the glquake code and the QuakeWorld code? Or we had gotten Paul to build us all new models? Or we had made all new maps that hang together thematically?
If we had, we would be releasing Quake right about now. It would be a much better game (it would be Quake 2), but all of the enjoyment that everyone has gotten from Quake would have been lost. It would have been the wrong decision.
Quake 2 is great, and it will get better yet after its release.
A reminder about "John Carmacks":
Anyone claiming to be me on IRC is lying. I have never been on IRC, and if I ever choose to, I will mention it here first.
If you get an unsolicited email from "John Carmack", the odds are high that it was spoofed. Every couple days, I get a mail bounce from someone who messed up on a spoofed mail, and I often get confused responses from people that I have never mailed.
-----------------------------------------
John Carmack's .plan for Dec 09, 1997
-----------------------------------------
BIG BUG IN Q2 NETWORKING!
If you run multiplayer servers, download:
ftp://ftp.idsoftware.com/idstuff/quake2/q2_306.zip
A serious bug got through.. I thought the QuakeWorld master server code was completely disabled, because I was planning on putting a modified architecture in place in the point release. It turns out that the code is still in there, sending heartbeats to a unix machine here at id that isn't even running a master server.
That wouldn't normally be an issue - a packet every five minutes from all the servers.
Except...
Cyrix has a new processor that is significantly faster at single precision floating point calculations if you don't do any double precision calculations anywhere.
Quake had always kept its timebase as a double precision seconds value, but I agreed to change it over to an integer millisecond timer to allow the global setting of single precision mode.
We went through and changed all the uses of it that we found, but the routine that sends heartbeats to the master servers was missed.
So, instead of sending a packet every 300 seconds, it is sending one every 300 MILLISECONDS.
Oops.
To a server, it won't really make a difference. A tiny extra packet three times a second is a fraction of the bandwidth of a player.
However, if there are thousands of network games in progress, that is a LOT of packets flooding idsoftware.com.
So, please download the new executable if you are going to run any servers (even servers started through the menus).
ftp://ftp.idsoftware.com/idstuff/quake2/q2_306.zip
This isn't the real point release - there are no new features or bugfixes. I just went back to the release codebase and recompiled with one function commented out so we wouldn't have to worry about introducing new bugs with our current untested code.
Btw, all bug reports should go to Christian (xian@idsoftware.com), NOT to me, Brian, or Cash! We need a central point to funnel things through. Hopefully we can set up a web page or something to make public what we know about, so we can cut down on email traffic.
-----------------------------------------
John Carmack's .plan for Dec 11, 1997
-----------------------------------------
The Quake 2 public code release is up at:
ftp://ftp.idsoftware.com/idstuff/quake2/source/q2source_12_11.zip
This source code distribution is only for hard-core people that are going to spend a lot of time pouring over it. This is NOT a how-to-make-levels-for-q2 type dsitribution!
This should keep a bunch of you busy for a while. :)
-----------------------------------------
John Carmack's .plan for Dec 23, 1997
-----------------------------------------
The DOOM source is up.
Merry christmas!
ftp://ftp.idsoftware.com/idstuff/source/doomsrc.zip
---- contents of README.TXT -----
Here it is, at long last. The DOOM source code is released for your non-profit use. You still need real DOOM data to work with this code. If you don't actually own a real copy of one of the DOOMs, you should still be able to find them at software stores.
Many thanks to Bernd Kreimeier for taking the time to clean up the project and make sure that it actually works. Projects tends to rot if you leave it alone for a few years, and it takes effort for someone to deal with it again.
The bad news: this code only compiles and runs on linux. We couldn't release the dos code because of a copyrighted sound library we used (wow, was that a mistake - I write my own sound code now), and I honestly don't even know what happened to the port that microsoft did to windows.
Still, the code is quite portable, and it should be straightforward to bring it up on just about any platform.
I wrote this code a long, long time ago, and there are plenty of things that seem downright silly in retrospect (using polar coordinates for clipping comes to mind), but overall it should still be a usefull base to experiment and build on.
The basic rendering concept - horizontal and vertical lines of constant Z with fixed light shading per band was dead-on, but the implementation could be improved dramatically from the original code if it were revisited. The way the rendering proceded from walls to floors to sprites could be collapsed into a single front-to-back walk of the bsp tree to collect information, then draw all the contents of a subsector on the way back up the tree. It requires treating floors and ceilings as polygons, rather than just the gaps between walls, and it requires clipping sprite billboards into subsector fragments, but it would be The Right Thing.
The movement and line of sight checking against the lines is one of the bigger misses that I look back on. It is messy code that had some failure cases, and there was a vastly simpler (and faster) solution sitting in front of my face. I used the BSP tree for rendering things, but I didn't realize at the time that it could also be used for environment testing. Replacing the line of sight test with a bsp line clip would be pretty easy. Sweeping volumes for movement gets a bit tougher, and touches on many of the challenges faced in quake / quake2 with edge bevels on polyhedrons.
Some project ideas:
Port it to your favorite operating system.
Add some rendering features - transparency, look up / down, slopes, etc.
Add some game features - weapons, jumping, ducking, flying, etc.
Create a packet server based internet game.
Create a client / server based internet game.
Do a 3D accelerated version. On modern hardware (fast pentium + 3DFX) you probably wouldn't even need to be clever - you could just draw the entire level and get reasonable speed. With a touch of effort, it should easily lock at 60 fps (well, there are some issues with DOOM's 35 hz timebase..). The biggest issues would probably be the non-power of two texture sizes and the walls composed of multiple textures.
I don't have a real good guess at how many people are going to be playing with this, but if significant projects are undertaken, it would be cool to see a level of community cooperation. I know that most early projects are going to be rough hacks done in isolation, but I would be very pleased to see a coordinated 'net release of an improved, backwards compatable version of DOOM on multiple platforms next year.
Have fun.
John Carmack 12-23-97
-----------------------------------------
John Carmack's .plan for Dec 25, 1997
-----------------------------------------
We are going to release a new quake 2 executable that fixes the malicious server crashing problems Real Soon Now. It also fixes a ton of other problems that have been reported, so we are going to have to give it some good testing before releasing it.
John Cash has two kids that would lynch him if he came in and worked on christmas, so we certainly won't be able to get a release candidate together before the weekend. I am fairly confidant we will have it released to the public on sunday.
I have been spending most of my time on trinity research but I have still made quite a few fixes to Q2. John Cash has made many more (he is just finishing up the IPX coding, among other things).
I have been doing a lot of testing over a proxy that gives me a very bad ping (400 - 800), so I was able to find and fix two significant errors with the prediction code.
The reason why you get a jerk when running forward and firing rockets, blasters, or grenades is that the client side prediction code was blocking you on your own missiles.
The jerky behavior on plats was due to a subtle error in the prediction error interpolation. A prediction error was causing oscillations as long as your latency, instead of smoothing out over just 100 ms. The plats are now smooth as long as you aren't dropping packets, and other mispredictions are also handled much better.
There are still a lot of other things that will be fixed in an upcoming release, but this will definately be an executable worth grabbing.
My fixes:
* zombies aren't being removed properly
* joystick not in menu
* classname for rockets and bolts
* no screaming when invulnerable and in lava
* lowered water blend values
* clear powerups when dead (no more breather sounds)
* only play "computer updated" three times max
* mapname serverinfo now updated properly
* changed "rejected a connection" to "Server is full"
* made console "rejected a connection" a developer only message
* made WSAWOULDBLOCK warning silent
* max 10 packets/second during connection process
* set cl_maxfps to 90
* increased loading plaque timeout value to 120 seconds
* paused not default to 1
* no savegame in deathmatch
* fixed ; binding from menu
* no crouch when airborne
* removed half-baked $ macro expansion
* pause on landing before re-jump (fixes no fall damage bug)
* public server framework
* no ; comment in config files
* teleporter events
* lower hyperblaster damage
* don't use PORT_ANY for clients!
* fix the entity number thing here
* don't re-check CD after the first time
* auto cddir from cd scan
* dissallow kill from intermissions
* faster rockets
* less bfg effect damage
* remove packet command from client
* strip trailing spaces on cmd_args
* added protocol to serverinfo
* used CMD_BACKUP instead of UPDATE_BACKUP for phone jack
* don't predict clip into your own missiles
* good netgraph
* validate userinfo for semicolons and quotes
* don't copy savegames on dedicated servers
* also check current directory for game dll loading
* changed connect packet on client to differ from server
* bump protocol version
* fixed error interpolation on plats
* only respawn with attack or jump
* fov as a userinfo
* show weapon icon if fov > 90
-----------------------------------------
John Carmack's .plan for Dec 27, 1997
-----------------------------------------
The 1.07 patch is out:
ftp://ftp.idsoftware.com/idstuff/quake2/patch_07.zip
Please mirror and distribute this.
When submitting bugs, make sure you say that you already have the 3.07 patch.
Christian will go through and update the bug page when he gets back from vacation next week.
This release does not fix all known problems. We intend to have another release in a few weeks.
-----------------------------------------
John Carmack's .plan for Dec 28, 1997
-----------------------------------------
** Dec 28, 2:55 AM CST **
There were a few problems with the 1.07 patch:
Bodies stuck under doors caused a repeated explosion effect. Timedemo was broken. The servers crash about once an hour under full load.
I have the first two fixed, and I hope the third. The four servers at Id are running a new executable.
If the servers don't crash in the next hour or two, I'll put another release out.
** Dec 28, 5:00 AM CST **
Ok, two hours without a crash on four servers.
Here is a new patch:
ftp://ftp.idsoftware.com/idstuff/quake2/patch_08.zip
3.07 and 3.08 can interoperate fine. All servers should upgrade to 3.08, but if you gravved the 3.07 earlier today and only play as a client and don't need timedemo, you don't nned to upgrade.
** Dec 28, 5:00 PM CST **
No crashes on any of the servers!
A few comments on some reported problems:
You have to press the "attack" button to respawn in deathmatch now. This allows you to chat and go into the menu. I have got several mails from people that are typing "kill" or reconnecting to servers after they die...
Old savegames will NOT work with the patch. Just cheat yourself to aproximately the same place you were before. The game included config files for starting off at each unit. You can exec one of those to get you close, then do "give" commands if you want to be more precise. (bigguun.cfg, boss.cfg city.cfg, command.cfg, factory.cfg, hangar.cfg, jail.cfg, mine.cfg, power.cfg, space.cfg, warehouse.cfg).
I think several people are failing to get the gamex86.dll into the baseq2 directory. if "fov 120" doesn't change your field of view, the server doesn't have the right gamex86.dll.
** Dec 28, 9:30 PM CST **
If Quake2 is crashing on you after upgrading, it is probably because you still have the gamex86.dll from q2test in your quake2 directory. The latest quake2.exe just started looking in the exe directory as well as the game directory to make debugging easier, and it brought out this problem. You should only have gamex86.dll in baseq2 unless you are doing specific development.
I had a version check in there, but I never bumped the game api version, so it was ineffective.
We are going to release yet another new version tonight.
The big news is that the modem connection and level changing problems are fixed. They should have been fixed in 3.07, but a timing error kept it from functioning.
I also found the "no such frame" warnings that scrolled by under some circumstances. BFG gibbing crouching people would cause it.
There are several other fixes in the menu and renderers as well, so everyone should upgrade.
We are testing with 3.09 on our servers now, but I want to make an incompatable change before releasing:
Right now, any client can send a "connect" message to the server and grab a client slot. If they are the wrong version, they will tie that slot up until they time out ot abort the connection process.
I am going to force clients to send their version number with the connection request, so that bad clients will never take up slots.
That will require everyone to upgrade to 3.09 to play.
I apologize for the flurry of versions, but this was a forced set of releases due to the server attacks, and lots of people are on vacation here. It certainly could have been tested better, but I thought it better to try and get something out ASAP.
Check back in the morning for a new version...
BTW, we will release the new gamex86 source code after we are convinced that we aren't going to be making another patch for a couple weeks.
-----------------------------------------
John Carmack's .plan for Dec 29, 1997
-----------------------------------------
** Dec 29, 2:25 AM CST **
new version:
ftp://ftp.idsoftware.com/idstuff/quake2/q2-309.exe
This one has an install that makes sure things get where they need to...
** Dec 29: 2:30 PM CST **
The only widely reported problem with 3.09 is that the video playback is choppy. The fix for the modem connections reduced video playback to 10 fps. Its a one line fix, but I'll hold off on another version until a few more things accumulate.
I am curious what the breakdown of opinion is on the rapid patch releases. If one of the polling websites would pose the question, I would apreciate it.
A more liesurely patch release would allow us more testing time, and some problems (like this cinematic bug) would certainly be killed before the public saw it, but I definately found a couple things from the public that no amount of testing on our machines would have found. Some things only showed up with 48 people playing on our servers for several hours.
Once again, we really didn't have a choice this time because of the server crashers, but we are planning another release in two to three weeks.
I am happy to produce new versions fairly rapidly, rather than at monthly intervals, but I know that many people are getting a little irate at having to download new patches. There is a simple solution -- if you don't want to be on the bleeding edge, wait a week after a patch is announced and see how it is working for other people.
What finally helped me get to the bottom of some things was just getting people with problems we couldn't reproduce to call me and let me send them executables by email until I figured out what was going on. From now on, if you send a detailed problem to me, include a phone number and times when you can be reached. I'm not tech support, so you certainly can't count on a response, but if you have a nice repeatable case of a problem that is high priority for us that we can't reproduce otherwise, your personal help may be usefull.
BTW, does anyone know why Quake 2 became a hacker target? I can keep fighting attacks, but spending my time there doesn't help anyones game, and there are a bunch of things that fundamentally can't be stopped if people really set their mind to messing up the servers.
** Dec 29: 8:00 PM CST **
Please cool it a bit with the email to me unless it is really important. I'll never get trinity done with the email pouring in the way it is right now...
We have rebuilt the 3.09 patch with a new version of the install program. Some people were not able to run the installer because a temp directory wasn't setup correctly. There are NO OTHER CHANGES in this, so if you were able to install the last 3.09, don't bother getting this one.
ftp://ftp.idsoftware.com/idstuff/quake2/q2-309-2.exe
-----------------------------------------
John Carmack's .plan for Dec 30, 1997
-----------------------------------------
Until we release the new gamex86 source code, if you want to make mods work with 3.09, change GAME_API_VERSION to:
#define GAME_API_VERSION 2
and recompile the mod.
This will let it run with the 3.09 servers. The API didn't actually change, I just had to bump that version number so that we could detect the old q2test dlls still hanging around.
-----------------------------------------
John Carmack's .plan for Dec 31, 1997
-----------------------------------------
** Dec 31: 4:40 AM: **
A user just reported having their net quake problems go away when they killed ICQ. I suppose it has never been stated directly, so here goes:
Quake needs all the bandwidth that a modem connection provides to play well. Any other program accessing the internet is going to cause a degredation in gameplay, sometimes severe.
So quit IRC, ICQ, email, and web browsers before setting out for serious net play unless you have ISDN or better.
** Dec 31: 12:41 PM CST: **
I just spent a few hours working with a quake player that still couldn't net quake with 3.09.
It took a while, but I finally understand what is going on.
He could play net games on his local lan, but when he tried to connect to remote servers, it would always fail and timeout midway through the connection process, or at most a few seconds into the game.
The situation was that there was a small network of computers connected to an ISDN router that did address translation.
Address translation allows multiple computers to use the internet through a single TCP/IP address. This is accomplished by having the router perform some "invisible" port and ip renaming on everything that goes out.
I think that is a rather evil thing for a router to do, but I suppose I can see the incentive from an address pressure viewpoint.
Routers know when TCP streams begin and end, so they make sure the port mappings stay constant through the entire thing, but quake uses UDP packets (anyone who suggests using TCP for a realtime game does not understand how the error recovery works), and the router apears to be making the incorrect assumption that UDP is only used for simple request / response protocols.
The router changes the UDP port while you are playing.
Grrrr.
Now, a smarter router would only change the port numbers when it was actually forced to by a collision, which would only be when a connection was first opened, and everything would work out ok.
After I understood what was happening, I could devise a fix for it. My simple fix was to make the server simply ignore the port number for client comparisons, and assume that if a packet came from the same IP address, then it is the same player even if the port number changed. That worked, and he was able to connect in to my modified server.
That has the distinct drawback of making translating routers or proxies that do the port mapping correctly unusable by more than one player at a time.
I could fix it completely by including a sort of port number in each message, and having the servers match and update UDP ports based on that. That would work fine, but at the cost of adding a byte or two to everyone's packets to help out people with bad routers. You wouldn't be able to tell a difference, but its the principle of it...
I could make a server side cvar to force port fixing on, but that would still not work for one class of users or the other.
I could make it client settable and have the client tell the server on connect that it needs port fixing. That would work with no bandwidth cost to anyone, but it would require users to know that if they can't connect to servers, then they should try to use the fix translation option. Unfortunately, I bet that there are some routers that exhibit this problem much less often. A drop every ten minutes would be hard to attribute.
I could make port fixing on by default, but if anyone is on a translated lan and another person tries to start a net quake game to the same server then they will both collide and crash and burn.
I am probably going to add the extra bytes to every packet. Being automatically robust on more people's systems is probably worth a microscopic loss of bandwidth. Two bytes is under one millisecond of ping on a modem.
If there is some magic range of port values that I can use to make these routers act better, let me know.
These changes will break the connection protocol again, so I am going to hold off on the patch for a while.
-----------------------------------------
John Carmack's .plan for Jan 01, 1998
-----------------------------------------
Some of the things I have changed recently:
* fixed the cinematics
* don't clear config after dedicated server
* don't reallocate sockets unless needed
* don't process channel packets while connecting
* rate variable for modem bandwidth choking
* delta compress client usercmds
* fixed sound quality changing after intermissions
* fixed PVS problem when head was directly under solid in GL
* added r_drawflat and cl_testlights to cheats
There are a few problems that I am still trying to track down:
WSAEADDRNOTAVAIL errors
Map versions differ error
Sometimes connecting and seeing messages but not getting in
Decompression read overrun.
Of course, we don't actually get any of those errors on any of our systems here, so I am having to work remotely with other users to try and fix them, which is a bit tougher.
My new years resolution is to improve my coding style by bracing all single line statements and consistantly using following caps on multi word variable names.
Actually, I am currently trying on the full sun coding style, but I'm not so sure about some of the comment conventions: don't use multiple lines of // comments, and don't use rows of seperating characters in comments. I'm not convinced those are good guidelines.
-----------------------------------------
John Carmack's .plan for Jan 02, 1998
-----------------------------------------
Wired magazine does something that almost no other print magazine we have dealt with does.
They check the statements they are going to print.
I just got a "fact check" questionaire email from wired about an upcoming article, and I recall that they did this last time the did an article about us.
Most of the time when we talk with the press, we try to get them to send us a proof of the article for fact checking. They usually roll their eyes, and grudgingly agree, then don't send us anything, or send it to us after it has gone to press.
Wired had a few errors in their statements, but it won't get printed that way because they checked with us.
How refreshing.
--
A small public announcement:
The Linux Expo is looking for:
1. People that develop games or game servers in *nix, and 2. People interested in learning how to develop games in *nix.
Either one should give a write to ddt@crack.com.
-----------------------------------------
John Carmack's .plan for Jan 03, 1998
-----------------------------------------
New stuff fixed:
* timeout based non-active packet streams
* FS_Read with CD off checks
* dedicated server not allocate client ports
* qport proxy checking stuff
* fixed mouse wheel control
* forced newlines on several Cbuf_AddText ()
* if no nextmap on a level, just stay on same one
* chat maximums to prevent user forced overflows
* limit stringcmds per frame to prevent malicious use
* helped jumping down slopes
* checksum client move messages to prevent proxy bots
* challenge / response connection process
* fixed rcon
* made muzzle flash lights single frame, rather than 0.1 sec
I still don't have an answer to the WAADRNOTAVAILABLE problem. I have made the packet stream as friendly as possible, but some computers are still choking.
I managed to get fixes for address translating routers done without costing any bandwidth from the server, just a couple bytes from the client, which isn't usually a critical path.
I have spent a fair amount of time trying to protect against "bad" users in this release. I'm sure there will be more things that come up, but I know I got a few of the ones that are currently being exploited.
We will address any attack that can make a server crash. Other attacks will have to have the damage and prevelence weighed against the cost of defending against it.
Client message overflows. The maximum number of commands that can be issued in a user packet has been limited. This prevents a client from doing enough "says" or "kills" to overflow the message buffers of other clients.
Challenge on connection. A connection request to a server is now a two stage process of requesting a challenge, then using it to connect. This prevents denial of service attacks where connection packets with forged IPs are flooded at a server, preventing any other users from connecting until they timeout.
Client packet checksumming. The packets are encoded in a way that will prevent proxies that muck with the packet contents, like the stoogebot, from working.
-----------------------------------------
John Carmack's .plan for Jan 04, 1998
-----------------------------------------
Version 3.10 patch is now out.
ftp://ftp.idsoftware.com/idstuff/quake2/q2-310.exe
A few more minor fixes since yesterday:
* qhost support
* made qport more random
* fixed map reconnecting
* removed s_sounddir
* print out primary / secondary sound buffer status on init
* abort game after a single net error if not dedicated
* fixed sound loss when changing sound compatability
* removed redundant reliable overflow print on servers
* gl_lockpvs for map development checking
* made s_primary 0 the default
Christian will be updating the bug page tomorrow. So hold of on all reporting for 24 hours, then check the page to make sure the bug is not already known.
http://www.idsoftware.com/cgi-win/bugs.exe
All bug reports should go to Christian: xian@idsoftware.com.
I have had several cases of people with lockup problems and decompression overreads having their problems fixed after they mentioned that they were overclocking either their CPU, their system bus (to 75mhz), or their 3DFX.
It doesn't matter if "it works for everything else", it still may be the source of the problem.
I know that some people are still having problems with vanilla systems, though. I have tried everything I can think of remotely, but if someone from the Dallas area wants to bring a system by our office, I can try some more serious investigations.
Something that has shown to help with some 3dfx problems is to set "cl_maxfps 31", which will keep the console between level changes from rendering too fast, which has caused some cards to hang the system.
-----------------------------------------
John Carmack's .plan for Jan 09, 1998
-----------------------------------------
We got 70 people on a base100 server, and it died after it wedged at 100% utilization for a while. Tomorrow we will find exactly what overflowed, and do some profiling.
Base100 is really only good for 50 or so players without overcrowding, but we have another map being built that should hold 100 people reasonably well.
I will look into which will be the easier path to more server performance: scalar optimization of whatever is critical now, or splitting it off into some more threads to run on multiple processors. Neither one is trivial.
My goal is to be able to host stable 100 player games in a single map.
I just added a "players" command that will dump the total number of players in the game, and as many frags/names as it can fit in a packet (around 50, I think).
-----------------------------------------
John Carmack's .plan for Jan 11, 1998
-----------------------------------------
I AM GOING OUT OF TOWN NEXT WEEK, DON'T SEND ME ANY MAIL!
Odds are that I will get back and just flush the 500 messages in my mailbox.
No, I'm not taking a vacation. Quite the opposite, in fact.
I'm getting a hotel room in a state where I don't know anyone, so I can do a bunch of research with no distractions.
I bought a new computer specifically for this purpose - A Dolch portable pentium-II system. The significant thing is that it has full length PCI slots, so I was able to put an Evans & Sutherland OpenGL accelerator in it (not enough room for an intergraph Realizm, though), and still drive the internal LCD screen. It works out pretty well, but I'm sure there will be conventional laptops with good 3D acceleration available later this year.
This will be an interesting experiment for me. I have always wondered how much of my time that isn't at peak productivity is a necessary rest break, and how much of it is just wasted.
---
The client's IP address is now added to the userinfo before calling ClientConnect(), so any IP filtering / banning rules can now be implemented in the game dll. This will also give some of you crazy types the ability to sync up with multiple programs on the client computers outside of Q2 itself.
A new API entry point has been added to the game dll that gets called whenever an "sv" command is issued on the server console. This is to allow you to create commands for the server operator to type, as opposed to commands that a client would type (which are defined in g_cmds.c).
---
We did a bunch of profiling today, and finaly got the information I wanted. We weren't doing anything brain dead stupid in the server, and all of the time was pretty much where I expected it to be.
I did found two things we can pursue for optimization.
A moderately expensive catagorization function is called at both the beginning and end of client movement simulation. With some care, we should be able to avoid the first one most of the time. That alone should be good for a >10% server speedup.
The other major thing is that the client movement simulation accounted for 60% of the total execution time, and because it was already compartmentalized for client side prediction, it would not be much work to make it thread safe. Unfortunately, it would require MAJOR rework of the server code (and some of the game dll) to allow multiple client commands to run in parallel.
The potential is there to double the peak load that a server can carry if you have multiple processors. Note that you will definately get more players / system by just running multiple independent servers, rather than trying to get them all into a single large server.
We are not going to pursue either of these optimizations right now, but they will both be looked at again later.
All this optimizing of the single server is pushing the tail end of a paradigm. I expect trinity to be able to seamlessly hand off between clustered servers without the client even knowing it happened.
-----------------------------------------
John Carmack's .plan for Feb 04, 1998
-----------------------------------------
Ok, I'm overdue for an update.
The research getaway went well. In the space of a week, I only left my hotel to buy diet coke. It seems to have spoiled me a bit, the little distractions in the office grate on me a bit more since. I will likely make week long research excursions a fairly regular thing during non- crunch time. Once a quarter sounds about right.
I'm not ready to talk specifically about what I am working on for trinity. Quake went through many false starts (beam trees, portals, etc) before settling down on its final architecture, so I know that the odds are good that what I am doing now won't actually be used in the final product, and I don't want to mention anything that could be taken as an implied "promise" by some people.
I'm very excited by all the prospects, though.
Many game developers are in it only for the final product, and the process is just what they have to go through to get there. I respect that, but my motivation is a bit different.
For me, while I do take a lot of pride in shipping a great product, the achievements along the way are more memorable. I don't remember any of our older product releases, but I remember the important insights all the way back to using CRTC wraparound for infinate smooth scrolling in Keen (actually, all the way back to understanding the virtues of structures over parallel arrays in apple II assembly language..). Knowledge builds on knowledge.
I wind up catagorizing periods of my life by how rich my learning experiences were at the time.
My basic skills built up during school on apple II computers, but lack of resources limited how far and fast I could go. The situation is so much better for programmers today - a cheap used PC, a linux CD, and an internet account, and you have all the tools and resources necessary to work your way to any level of programming skill you want to shoot for.
My first six months at Softdisk, working on the PC, was an incredible learning experience. For the first time, I was around a couple of programmers with more experience than I had (Romero and Lane Roath), there were a lot of books and materials available, and I could devote my full and undivided attention to programming. I had a great time.
The two years following, culminating in DOOM and the various video game console work I did, was a steady increase in skills and knowledge along several fronts - more graphics, networking, unix, compiler writing, cross development, risc architectures, etc.
The first year of Quake's development was awesome. I got to try so many new things, and I had Michael Abrash as my sounding board. It would probably surprise many classically trained graphics programmers how little I new about conventional 3D when I wrote DOOM - hell, I had problems properly clipping wall polygons (which is where all the polar coordinate nonsense came from). Quake forced me to learn things right, as well as find some new innovations.
The last six months of Quake's development was mostly pain and suffering trying to get the damn thing finished. It was all worth it in the end, but I don't look back at it all that fondly.
The development cycle of Quake 2 had some moderate learning experiences for me (glquake, quakeworld, radiosity, openGL tool programming, win32, etc), but it also gave my mind time to sift through a lot of things before getting ready to really push ahead.
I think that the upcoming development cycle for trinity is going to be at least as rewarding as Quake's was. I am reaching deep levels of understanding on some topics, and I am branching out into several completely new (non-graphics) areas for me, that should cross-polinate well with everything else I am doing.
There should also be a killer game at the end of it. :)
-----------------------------------------
John Carmack's .plan for Feb 09, 1998
-----------------------------------------
Just got back from the Q2 wrap party in vegas that Activision threw for us.
Having a reasonable grounding in statistics and probability and no belief in luck, fate, karma, or god(s), the only casino game that interests me is blackjack.
Playing blackjack properly is a test of personal discipline. It takes a small amount of skill to know the right plays and count the cards, but the hard part is making yourself consistently behave like a robot, rather than succumbing to your "gut instincts".
I play a basic high/low count, but I scale my bets widely - up to 20 to 1 in some cases. Its not like I'm trying to make a living at it, so the chance of getting kicked out doesn't bother me too much.
I won $20,000 at the tables, which I am donating to the Free Software Foundation. I have been meaning to do something for the FSF for a long time. Quake was deployed on a dos port of FSF software, and both DOOM and Quake were developed on NEXTSTEP, which uses many FSF based tools. I don't subscribe to all the FSF dogma, but I have clearly benefited from their efforts.
-----------------------------------------
John Carmack's .plan for Feb 12, 1998
-----------------------------------------
I have been getting a lot of mail with questions about the intel i740 today, so here is a general update on the state of 3D cards as they relate to quake engine games.
ATI rage pro
----
On paper, this chip looks like it should run almost decently - about the performance of a permedia II, but with per-pixel mip mapping and colored lighting. With the currently shipping MCD GL driver on NT, it just doesn't run well at all. The performance is well below acceptable, and there are some strange mip map selection errors. We have been hearing for quite some time that ATI is working on an OpenGL ICD for both '95 and NT, but we haven't seen it yet. The rage pro supposedly has multitexture capability, which would help out quite a bit if they implement the multitexture extension. If they do a very good driver, the rage pro may get up to the performance of the rendition cards. Supports up to 16MB, which would make it good for development work if the rest of it was up to par.
3DLabs permedia II
------
Good throughput, poor fillrate, fair quality, fair features.
No colored lighting blend mode, currently no mip mapping at all.
Supports up to 8MB.
The only currently shipping production full ICD for '95, but a little flaky.
If 3dlabs implemented per-polygon mip mapping, they would get both a quality and a slight fillrate boost.
Drivers available for WinNT on the DEC Alpha (but the alpha drivers are very flaky).
Power VR PCX2
-----
Poor throughput, good fillrate, fair quality, poor features, low price.
No WinNT support.
Almost no blend modes at all, low alpha precision.
Even though the hardware doesn't support multitexture, they could implement the multi-texture extension just to save on polygon setup costs. That might get them a 10% to 15% performance boost.
They could implement the point parameters extension for a significant boost in the speed of particle rendering. That wouldn't affect benchmark scores very much, but it would help out in hectic deathmatches.
Their opengl minidriver is already a fairly heroic effort - the current PVR takes a lot of beating about the head to make it act like an OpenGL accelerator.
Rendition v2100 / v2200
--------
Good throughput, good fillrate, very good quality, good features.
A good all around chip. Not quite voodoo1 performance, but close.
v2100 is simply better than everything else in the $99 price range.
Can render 24 bit color for the best possible quality, but their current drivers don't support it. Future ones probably will.
Can do 3D on the desktop.
Rendition should be shipping a full ICD OpenGL, which will make an 8mb v2200 a very good board for people doing 3D development work.
NVidia Riva 128
-----
Very good throughput, very good fillrate, fair quality, fair features.
The fastest fill rate currently shipping, but it varies quite a bit based on texture size. On large textures it is slightly slower than voodoo, but on smaller textures it is over twice as fast.
On paper, their triangle throughput rate should be three times what voodoo gives, but in practice we are only seeing a slight advantage on very fast machines, and worse performance on pentium class machines. They probably have a lot of room to improve that in their drivers.
In general, it is fair to say that riva is somewhat faster than voodoo 1, but it has a few strikes against it.
The feature implementation is not complete. They have the blend mode for colored lighting, but they still don't have them all. That may hurt them in future games. Textures can only be 1 to 1 aspect ratio. In practice, that just means that non-square textures waste memory.
The rendering quality isn't quite as high as voodoo or rendition. It looks like some of their iterators don't have enough precision.
Nvidia is serious and committed to OpenGL. I am confident that their driver will continue to improve in both performance and robustness.
While they can do good 3D in a window, they are limited to a max of 4MB of framebuffer, which means that they can't run at a high enough resolution to do serious work.
3DFX Voodoo 1
-----
The benchmark against which everything else is measured.
Good throughput, good fillrate, good quality, good features.
It has a couple faults, but damn few: max texture size limited to 256*256 and 8 to 1 aspect ratio. Slow texture swapping. No 24 bit rendering.
Because of the slow texture swapping, anyone buying a voodoo should get a six mb board (e.g. Canopus Pure3D). The extra ram prevents some sizable jerks when textures need to be swapped.
Highly tuned minidriver. They have a full ICD in alpha, but they are being slow about moving it into production. Because of the add-in board nature of the 3dfx, the ICD won't be useful for things like running level editors, but it would at least guarantee that any new features added to quake engine games won't require revving the minidriver to add new functionality.
3DFX Voodoo 2
-----
Not shipping yet, but we were given permission to talk about the benchmarks on their preproduction boards.
Excellent throughput, excellent fillrate, good quality, excellent features.
The numbers were far and away the best ever recorded, and they are going to get significantly better. On quake 2, voodoo 2 is setup limited, not fill rate limited. Voodoo 2 can do triangle strip and fan setup in hardware, but their opengl can't take advantage of it until the next revision of glide. When that happens, the number of vertexes being sent to the card will drop by HALF. At 640*480, they will probably become fill rate bound again (unless you interleave two boards), but at 512*384, they will probably exceed 100 fps on a timedemo. In practice, that means that you will play the game at 60 fps with hardly ever a dropped frame.
The texture swapping rate is greatly improved, addressing the only significant problem with voodoo.
I expect that for games that heavily use multitexture (all quake engine games), voodoo 2 will remain the highest performer for all of '98. All you other chip companies, feel free to prove me wrong. :)
Lack of 24 bit rendering is the only visual negative.
As with any voodoo solution, you also give up the ability to run 3D applications on your desktop. For pure gamers, that isn't an issue, but for hobbyists that may be interested in using 3D tools it may have some weight.
Intel i740
----
Good throughput, good fillrate, good quality, good features.
A very competent chip. I wish intel great success with the 740. I think that it firmly establishes the baseline that other companies (especially the ones that didn't even make this list) will be forced to come up to.
Voodoo rendering quality, better than voodoo1 performance, good 3D on a desktop integration, and all textures come from AGP memory so there is no texture swapping at all.
Lack of 24 bit rendering is the only negative of any kind I can think of.
Their current MCD OpenGL on NT runs quake 2 pretty well. I have seen their ICD driver on '95 running quake 2, and it seems to be progressing well. The chip has the potential to outperform voodoo 1 across the board, but 3DFX has more highly tuned drivers right now, giving it a performance edge. I expect intel will get the performance up before releasing the ICD.
It is worth mentioning that of all the drivers we have tested, intel's MCD was the only driver that did absolutely everything flawlessly. I hope that their ICD has a similar level of quality (it's a MUCH bigger job).
An 8mb i740 will be a very good setup for 3D development work.
-----------------------------------------
John Carmack's .plan for Feb 16, 1998
-----------------------------------------
8 mb or 12 mb voodoo 2?
An 8mb v2 has 2 mb of texture memory on each TMU. That is not as general as the current 6mb v1 cards that have 4 mb of texture memory on a single TMU. To use the multitexture capability, textures are restricted to being on one or the other TMU (simplifying a bit here). There is some benefit over only having 2 mb of memory, but it isn't double. You will see more texture swapping in quake on an 8mb voodoo 2 than you would on a 6mb voodoo 1. However, the texture swapping is several times faster, so it isn't necessarily all that bad.
If you use the 8 bit palettized textures, there will probably not be any noticable speed improvement with a 12 mb voodoo 2 vs an 8 mb one. The situation that would most stress it would be an active deathmatch that had players using every skin. You might see a difference there.
A game that uses multitexture and 16 bit textures for everything will stress a 4/2/2 voodoo layout. Several of the Quake engine licensees are using full 16 bit textures, and should perform better on a 4/4/4 card.
The differences probably won't show as significant on timedemo numbers, but they will be felt as little one frame hitches here and there.
-----------------------------------------
John Carmack's .plan for Feb 17, 1998
-----------------------------------------
I just read the Wired article about all the Doom spawn.
I was quoted as saying "like I'm supposed to be scared of Monolith", which is much more derogatory sounding than I would like.
I haven't followed Monolith's development, and I don't know any of their technical credentials, so I am not in any position to evaluate them.
The topic of "is microsoft going to crush you now that they are in the game biz", made me a bit sarcastic.
I honestly wish the best to everyone pursuing new engine development.
-----------------------------------------
John Carmack's .plan for Feb 22, 1998
-----------------------------------------
Don't send any bug reports on the 3.12 release to me, I just forward them over to jcash. He is going to be managing all future work on the Quake 2 codebase through the mission packs. I'm working on trinity.
3.12 answered the release question pretty decisively for me. We were in code freeze for over two weeks while the release was being professionally beta tested, and all it seemed to get us was a two week later release.
Future releases are going to be of the fast/multiple release type, but clearly labeled as a "beta" release until it stabilizes. A dozen professional testers or fifty amature testers just can't compare to the thousands of players who will download a beta on the first day.
I have spent a while thinking about the causes of the patches for Q2. Our original plan was to just have the contents of 3.12 as the first patch, but have it out a month earlier than we did.
The first several patches were forced due to security weaknesses. Lesson learned - we need to design more security conscious to try to protect against the assholes out there.
The cause for the upcoming 3.13 patch is the same thing that has caused us a fair amount of trouble through Q2's development - instability in the gamex86 code due to its decending from QC code in Q1. It turns out that there were lots of bugs in the original QC code, but because of its safe interpreted nature (specifically having a null entity reference the world) they never really bothered anyone. We basically just ported the QC code to regular C for Q2 (it shows in the code) and fixed crash bugs as they popped up. We should have taken the time to redesign more for C's strengths and weaknesses.
-----------------------------------------
John Carmack's .plan for Mar 12, 1998
-----------------------------------------
American McGee has been let go from Id.
His past contributions include work in three of the all time great games (DOOM 2, Quake, Quake 2), but we were not seeing what we wanted.
-----------------------------------------
John Carmack's .plan for Mar 13, 1998
-----------------------------------------
The Old Plan:
The rest of the team works on an aggressive Quake 2 expansion pack while Brian and I develop tools and code for the entirely new Trinity generation project to begin after the mission pack ships.
The New Plan:
Expand the mission pack into a complete game, and merge together a completely new graphics engine with the quake 2 game / client / server framework, giving us Quake 3.
"Trinity" is basically being broken up into two phases: graphics and everything else. Towards the end of Quake 1's development I was thinking that we might have been better off splitting quake on those categories, but in reverse order. Doing client/server, the better modification framework, and qc, coupled with a spiced up DOOM engine (Duke style) for one game, then doing the full 3D renderer for the following game.
We have no reason to believe that the next generation development would somehow go faster than the previous, so there is a real chance that doing all of the Trinity technology at once might push game development time to a full two years for us, which might be a bit more than the pressure-cooker work atmosphere here could handle.
So, we are going to try an experiment.
The non-graphics things that I was planning for Trinity will be held off until the following project - much java integration with client downloadable code being one of the more significant aspects. I hope to get to some next generation sound work, but the graphics engine is the only thing I am committing to.
The graphics engine is going to be hardware accelerated ONLY. NO SOFTWARE RENDERER, and it won't work very well on a lot of current hardware. We understand fully that this is going to significantly cut into our potential customer base, but everyone was tired of working under the constraints of the software renderer. There are still going to be plenty of good quake derived games to play from other developers for people without appropriate hardware.
There are some specific things that the graphics technology is leveraging that may influence your choice of a 3D accelerator.
All source artwork is being created and delivered in 24 bit color. An accelerator that can perform all 3D work in 24 bit color will look substantially better than a 16 bit card. You will pay a speed cost for it, though.
Most of the textures are going to be higher resolution. Larger amounts of texture memory will make a bigger difference than it does on Quake 2.
Some key rendering effects require blending modes that some cards don't support.
The fill rate requirements will be about 50% more than Quake 2, on average. Cards that are fill rate limited will slow down unless you go to a lower resolution.
The triangle rate requirements will be at least double Quake 2, and scalable to much higher levels of detail on appropriate hardware.
Here are my current GUESSES about how existing cards will perform.
Voodoo 1 Performance will be a little slow, but it should look good and run acceptably. You will have to use somewhat condensed textures to avoid texture thrashing.
Voodoo 2 Should run great. Getting the 12 mb board is probably a good idea if you want to use the high resolution textures. The main rendering mode won't be able to take advantage of the dual TMU the same way quake 2 does, so the extra TMU will be used for slightly higher quality rendering modes instead of greater speed: trilinear / detail texturing, or some full color effects where others get a mono channel.
Permedia 2 Will be completely fill rate bound, so it will basically run 2/3 the speed that quake 2 does. Not very fast. It also doesn't have one of the needed blending modes, so it won't look very good, either. P2 does support 24 bit rendering, but it won't be fast enough to use it.
ATI Rage Pro It looks like the rage pro has all the required blending modes, but the jury is still out on the performance.
Intel I740 Should run good with all features, and because all of the textures come out of AGP memory, there will be no texture thrashing at all, even with the full resolution textures.
Rendition 2100/2200 The 2100 should run about the speed of a voodoo 1, and the 2200 should be faster. They support all the necessary features, and an 8 mb 2200 should be able to use the high res textures without a problem. The renditions are the only current boards that can do 24 bit rendering with all the features. It will be a bit slow in 24 bit mode, but it will look the best.
PVR PCX2 Probably won't run Quake 3. They don't have ANY of the necessary blending modes, so it can't look correct. Video Logic might decide to rev their minidriver to try to support it, but it is probably futile.
RIVA 128 Riva puts us in a bad position. They are very fast, but they don't support an important feature. We can crutch it up by performing some extra drawing passes, but there is a bit of a quality loss, and it will impact their speed. They will probably be a bit faster than voodoo 1, but not to the degree that they are in Quake 2.
Naturally, the best cards are yet to come (I won't comment on unreleased cards). The graphics engine is being designed to be scalable over the next few YEARS, so it might look like we are shooting a bit high for the first release, but by the time it actually ships, there will be a lot of people with brand new accelerators that won't be properly exploited by any other game.
-----------------------------------------
John Carmack's .plan for Mar 20, 1998
-----------------------------------------
Robert Duffy, the maintainer of Radiant QE4 is now "officially" in charge of further development of the editor codebase. He joins Zoid as a (part time) contractor for us.
A modified version of Radiant will be the level editor for Quake 3. The primary changes will be support for curved surfaces and more general surface shaders. All changes will be publicly released, either after Q3 ships or possibly at the release of Q3Test, depending on how things are going.
The other major effort is to get Radiant working properly on all of the 3D cards that are fielding full OpenGL ICDs. If you want to do level development, you should probably get an 8mb video card. Permedia II cards have been the mainstay for developers that can't afford intergraph systems, but 8mb rendition v2200 (thriller 3D) cards are probably a better bet as soon as their ICD gets all the bugs worked out.
-----------------------------------------
John Carmack's .plan for Mar 21, 1998
-----------------------------------------
I just shut down the last of the NEXTSTEP systems running at id.
We hadn't really used them for much of anything in the past year, so it was just easier to turn them off than to continue to administer them.
Most of the intel systems had already been converted to NT or 95, and Onethumb gets all of our old black NeXT hardware, but we have four nice HP 712/80 workstations that can't be used for much of anything.
If someone can put these systems to good use (a dallas area unix hacker), you can have them for free. As soon as they are spoken for, I will update my .plan, so check immediately before sending me email.
You have to come by our office (in Mesquite) and do a fresh OS install here before you can take one. There may still be junk on the HD, and I can't spend the time to clean them myself. You can run either NEXTSTEP 3.3 or HP/UX. These are NOT intel machines, so you can't run dos or windows. I have NS CD's here, but I can't find the original HP/UX CDs. Bring your own if that's what you want.
I'm a bit nostalgic about the NeXT systems -- the story in the Id Anthology is absolutely true: I walked through a mile of snow to the bank to pay for our first workstation. For several years, I considered it the best development environment around. It still has advantages today, but you can't do any accelerated 3D work on it.
I had high hopes for rhapsody, but even on a top of the line PPC, it felt painfully sluggish compared to the NT workstations I use normally, and apple doesn't have their 3D act together at all.
Its kind of funny, but even through all the D3D/OpenGL animosity, I think Windows NT is the best place to do 3D graphics development.
All gone!
--------------
Paul Magyar gets the last (slightly broken) one.
Bob Farmer gets the third.
Philip Kizer gets the second one.
Kyle Bousquet gets the first one.
3/21 pt 2
---------
I haven't given up on rhapsody yet. I will certainly be experimenting with the release version when it ships, but I have had a number of discouraging things happen. Twice I was going to go do meetings at apple with all relevent people, but the people setting it up would get laid off before the meetings happened. Several times I would hear encouraging rumors about various things, but they never panned out. We had some biz discussions with apple about rhapsody, but they were so incredibly cautious about targeting rhapsody for consumer apps at the expense of macos that I doubted their resolve.
I WANT to help. Maybe post-E3 we can put something together.
The SGI/microsoft deal fucked up a lot of the 3D options. The codebase that everyone was using to develop OpenGL ICDs is now owned by microsoft, so it is unlikely any of them will ever be allowed to port to rhapsody (or linux, or BeOS).
That is one of the things I stress over -- The Right Thing is clear, but its not going to happen because of biz moves. It would be great if ATI, which has video drivers for win, rhapsody, linux, and BeOS, could run the same ICD on all those platforms.
-----------------------------------------
John Carmack's .plan for Mar 26, 1998
-----------------------------------------
I haven't even seen the "BeOS port of Quake". Stop emailing me about aproving it. I told one of the Lion developers he could port it to BeOS in his spare time, but I haven't seen any results from it.
-
There is a public discussion / compilation going on at openquake for suggestions to improve technical aspects of quake 3:
http://www.openquake.org/q3suggest/
This is sooo much better than just dropping me an email when a thought hits you. There are many, many thousands of you out there, and there needs to be some filtering process so we can get the information efficiently.
We will read and evaluate everything that makes it through the discussion process. There are two possible reasons why features don't make it into our games - either we decide that the effort is better spent elsewhere, or we just don't think about it. Sometimes the great ideas are completely obvious when suggested, but were just missed. That is what I most hope to see.
When the suggestions involve engineering tradeoffs and we have to consider the implementation effort of a feature vs its benefits, the best way to convince us to pursue it is to specify EXACTLY what benefits would be gained by undertaking the work, and specifying a clean interface to the feature from the file system data and the gamex86 code.
We hack where necessary, but I am much more willing to spend my time on an elegant extension that has multiple uses, rather than adding api bulk for specific features. Point out things that are clunky and inelegant in the current implementation. Even if it doesn't make any user visible difference, restructuring api for cleanliness is still a worthwhile goal.
We have our own ideas about game play features, so we may just disagree with you. Even if you-and-all-your-friends are SURE that your suggestions will make the game a ton better, we may not think it fits with our overall direction. We aren't going to be all things to all people, and we don't design by committee.
-----------------------------------------
John Carmack's .plan for Apr 02, 1998
-----------------------------------------
Drag strip day!
Most of the id guys, John Romero from ION, and George and Alan from 3drealms headed to the Ennis dragstrip today.
Nobody broke down, and some good times were posted.
11.9 @ 122 John Carmack F40
12.2 @ 122 George Broussard custom turbo 911
12.4 @ 116 Brian Hook Viper GTS
13.4 @ 106 John Romero custom turbo testarossa
13.6 @ 106 Todd Hollenshead 'vette
13.9 @ 100 Paul Steed 911
14.0 @ 99 Tim Willits 911
14.3 @ 101 Bear Turbo Supra
14.4 @ 98 Alan Blum turbo rx-7
14.7 @ 92 Brandon James M3
15.3 @ 92 Christian Boxster
15.5 @ 93 Jen (Hook's Chick) Turbo Volvo
16.1 @ 89 Ms. Donna Mustang GT
17.4 @ 82 Anna (Carmack's Chick) Honda Accord
18.1 @ 75 Jennifer (Jim Molinets' Chick) Saturn
We had three significant no-shows for various reasons: my TR, Adrian's viper, and Cash's supercharged M3 were all in the shop.
-----------------------------------------
John Carmack's .plan for Apr 08, 1998
-----------------------------------------
Things are progressing reasonably well on the Quake 3 engine.
Not being limited to supporting a 320*240 256 color screen is very, very nice, and will make everyone's lives a lot easier.
All of our new source artwork is being done in 24 bit TGA files, but the engine will continue to load .wal files and .pcx files for developer's convenience. Each pcx can have its own palette now though, because it is just converted to 24 bit at load time.
Q3 is going to have a fixed virtual screen coordinate system, independant of resolution. I tried that back in the original glquake, but the fixed coordinate system was only 320*200, which was excessively low. Q2 went with a dynamic layout at different resolutions, which was a pain, and won't scale to the high resolutions that very fast cards will be capable of running at next year.
All screen drawing is now done assuming the screen is 640*480, and everything is just scaled as you go higher or lower. This makes laying out status bars and HUDs a ton easier, and will let us do a lot cooler looking screens.
There will be an interface to let game dlls draw whatever they want on the screen, precisely where they want it. You can suck up a lot of network bandwidth doing that though, so some care will be needed.
-
Going to the completely opposite end of the hardware spectrum from quake 3..
I have been very pleased with the fallout from the release of the DOOM source code.
At any given spot in design space, there are different paths you can take to move forward. I have usually chosen to try to make a large step to a completely new area, but the temptation is there to just clean up and improve in the same area, continuously polishing the same program.
I am enjoying seeing several groups pouring over DOOM, fixing it up and enhancing it. Cleaning up long standing bugs. Removing internal limitations. Orthogonalizing feature sets. Etc.
The two that I have been following closest are Team TNT's BOOM engine project, which is a clear headed, well engineered improvement on the basic DOOM technical decisions, and Bruce Lewis' glDoom project.
Any quakers feeling nostalgic should browse around:
http://www.doomworld.com/
-----------------------------------------
John Carmack's .plan for Apr 16, 1998
-----------------------------------------
F40 + $465,000 = F50
-----------------------------------------
John Carmack's .plan for Apr 17, 1998
-----------------------------------------
Yes, I bought an F50. No, I don't want a McLaren.
We will be going back to the dragstrip in a couple weeks, and I will be exercising both the F50 and the TR there. Cash's supercharged M3 will probably show some of the porsches a thing or two, as well.
I'll probably rent a road coarse sometime soon, but I'm not in too much of a hurry to run the F50 into the weeds.
My TR finally got put back together after a terrific nitrous explosion just before the last dragstrip. It now makes 1000.0 hp at the rear wheels. Contrast that with the 415 rear wheel hp that the F40 made. Of course, a loaded testarossa does weigh about 4000 lbs..
My project car is somewhat nearing completion. My mechanic says it will be running in six weeks, but mechanics can be even more optimistic than software developers. :) I'm betting on fall. It should really be something when completed: a carbon fiber bodied ferrari GTO with a custom, one-of-a kind billet alluminum 4 valve DOHC 5.2L V12 with twin turbos running around 30 lbs of boost. It should be good for quite a bit more hp than my TR, and the entire car will only weigh 2400 lbs.
---
The distance between a cool demo and production code is vast. Two months ago, I had some functional demos of several pieces of the Quake 3 rendering tech, but today it still isn't usable as a full replacement for ref_gl yet.
Writing a modern game engine is a lot of work.
The new architecture is turning out very elegent. Not having to support software rendering or color index images is helping a lot, but it is also nice to reflect on just how much I have learned in the couple years since the original Quake renderer was written.
My C coding style has changed for Quake 3, which is going to give me a nice way of telling at a glance which code I have or haven't touched since Quake 2. In fact, there have been enough evolutions in my style that you can usually tell what year I wrote a piece of code by just looking at a single function:
[code]
/*
=============
=
= Function headers like this are DOOM or earlier
=
=============
*/
/*
=============
Function Headers like this are Quake or later
=============
*/
{
// comments not indented were written on NEXTSTEP
// (quake 1)
// indented comments were written on
// Visual C++ (glquake / quakeworld, quake2)
}
for (testnum=0 ; testnum<4 ; testnum++)
{ // older coding style
}
for (testNumber = 0 ; testNumber < 4 ; testNumber++) {
// quake 3 coding style
}
[/code]
-----------------------------------------
John Carmack's .plan for Apr 22, 1998
-----------------------------------------
F50 pros and cons vs F40:
The front and rear views are definately cooler on the F50, but I think I like the F40 side view better. I haven't taken the top off the F50 yet, though (its supposed to be a 40 minute job..).
Adjustable front suspension. Press a button and it raises two inches, which means you can actually drive it up into strip malls. The F40 had to be driven into my garage at an angle to keep the front from rubbing. This makes the car actually fairly practical for daily driving.
Drastically better off idle torque. You have to rev the F40 a fair amount to even get it moving, and if you are moving at 2000 rpm in first gear, a honda can pull away from you until it starts making boost at 3500 rpm. The f50 has enough torque that you don't even need to rev to get moving, and it goes quite well by just flooring it after you are moving. No need to wreck a clutch by slipping it out from 4000 rpm.
Much nicer clutch. The F40 clutch was a very low-tech single disk clutch that required more effort than on my crazy TR with over twice the torque.
Better rearward visibility. The F40's lexan fastback made everything to your rear a blur.
Better shifting. A much smoother six speed than the F40's five speed.
Better suspension. Some bumps that would upset the F40 badly are handled without any problems.
Better aerodynamics. A flat underbody with tunnels is a good thing if you are going to be moving at very high speeds.
I beleive the F50 could probably lap a road coarse faster than the F40, but in a straight line, the F40 is faster. The F50 felt a fair amount slower, but I was chalking that up to the lack of non-linear turbo rush. Today I drove it down to the dyno and we got real numbers.
It only made 385 hp at the rear wheels, which is maybe 450 at the crank if you are being generous. The F40 made 415, but that was with the boost cranked up a bit over stock.
We're going to have to do something about that.
I'm thinking that a mild twin-turbo job will do the trick. Six pounds of boost should get it up to a health 500 hp at the rear wheels, which will keep me happy. I don't want to turn it into a science project like my TR, I just want to make sure it is well out of the range of any normal cars.
I may put that in line after my GTO gets finished.
-----------------------------------------
John Carmack's .plan for May 02, 1998
-----------------------------------------
The rcon backdoor was added to help the development of QuakeWorld (It is not present in Quake 1). At the time, attacking Quake servers with spoofed packets was not the popular sport it seems to have become with Quake 2, so I didn't think much about the potential for exploitation.
The many forced releases of Quake 2 due to hacker attacks has certainly taught me to be a lot more wary.
It was a convenient feature for us, but it turned out to be irresponsible. Sorry.
There will be new releases of QuakeWorld and Quake 2 soon.
-----------------------------------------
John Carmack's .plan for May 04, 1998
-----------------------------------------
Here are some notes on a few of the technologies that I researched in preparing for the Quake3/trinity engine. I got a couple months of pretty much wide open research done at the start, but it turned out that none of the early research actually had any bearing on the directions I finally decided on. Ah well, I learned a lot, and it will probably pay off at some later time.
I spent a little while doing some basic research with lummigraphs, which are sort of a digital hologram. The space requirements are IMMENSE, on the order of several gigs uncompressed for even a single full sized room. I was considering the possibility of using very small lumigraph fragments (I called them "lumigraphlets") as imposters for large clusters of areas, similar to aproximating an area with a texture map, but it would effectively be a view dependent texture.
The results were interesting, but transitioning seamlessly would be difficult, the memory was still large, and it has all the same caching issues that any impostor scheme has.
Another aproach I worked on was basically extending the sky box code style of rendering from quake 2 into a complete rendering system. Take a large number of environment map snapshots, and render a view by interpolating between up to four maps (if in a tetrahedral arangement) based on the view position.
A simple image based interpolating doesn't convey a sense of motion, because it basically just ghosts between seperate points unless the maps are VERY close together reletive to the nearest point visible in the images.
If the images that make up the environment map cube also contain depth values at some (generally lower) resolution, instead of rendering the environment map as six big flat squares at infinity, you can render it as a lot of little triangles at the proper world coordinates for the individual texture points. A single environment map like this can be walked around in and gives a sense of motion. If you have multiple maps from nearby locations, they can be easily blended together. Some effort should be made to nudge the mesh samples so that as many points are common between the maps as possible, but even a regular grid works ok.
You get texture smearing when occluded detail should be revealed, and if you move too far from the original camera point the textures blur out a lot, but it is still a very good effect, is completely complexity insensitive, and is aliasing free except when the view position causes a silhouette crease in the depth data.
Even with low res environment maps like in Quake2, each snapshot would consume 700k, so taking several hundred environment images throughout a level would generate too much data. Obviously there is a great deal of redundancy - you will have several environment maps that contain the same wall image, for instance. I had an interesting idea for compressing it all. If you ignore specular lighting and atmospheric effects, any surface that is visible in multiple environment maps can be represented by a single copy of it and perspective transformation of that image. Single image, transformations, sounds like.. fractal compression. Normal fractal compression only deals with affine maps, but the extension to projective maps seems logical.
I think that a certain type of game could be done with a technology like that, but in the end, I didn't think it was the right direction for a first person shooter.
There is a tie in between lummigraphs, multiple environment maps, specularity, convolution, and dynamic indirect lighting. Its nagging at me, but it hasn't come completely clear.
Other topics for when I get the time to write more:
Micro environment map based model lighting. Convolutions of environment maps by phong exponent, exponent of one with normal vector is diffuse lighting.
Full surface texture representation. Interior antaliasing with edge matched texels.
Octree represented surface voxels. Drawing and tracing.
Bump mapping, and why most of the aproaches being suggested for hardware are bogus.
Parametric patches vs implicit functions vs subdivision surfaces.
Why all analytical boundary representations basically suck.
Finite element radiosity vs photon tracing.
etc.
-----------------------------------------
John Carmack's .plan for May 17, 1998
-----------------------------------------
Here is an example of some bad programming in quake:
There are three places where text input is handled in the game: the console, the chat line, and the menu fields. They all used completely different code to manage the input line and display the output. Some allowed pasting from the system clipboard, some allowed scrolling, some accepted unix control character commands, etc. A big mess.
Quake 3 will finally have full support for international keyboards and character sets. This turned out to be a bit more trouble than expected because of the way Quake treated keys and characters, and it led to a rewrite of a lot of the keyboard handling, including the full cleanup and improvement of text fields.
A similar cleanup of the text printing hapened when Cash implemented general colored text: we had at least a half dozen different little loops to print strings with slightly different attributes, but now we have a generalized one that handles embedded color commands or force-to-color printing.
Amidst all the high end graphics work, sometimes it is nice to just fix up something elementary.
-----------------------------------------
John Carmack's .plan for May 19, 1998
-----------------------------------------
A 94 degree day at the dragstrip today. Several 3drealms and Norwood Autocraft folk also showed up to run. We got to weigh most of the cars on the track scales, which gives us a few more data points.
11.6 @ 125 Bob Norwood's ferrari P4 race car (2200 lbs)
11.9 @ 139 John Carmack's twin turbo testarossa (3815 lbs)
11.9 @ 117 Paul Steed's YZF600R bike
12.1 @ 122 John Carmack's F50 (3205 lbs)
12.3 @ 117 Brian's Viper GTS (3560 lbs)
13.7 @ 103 John Cash's supercharged M3
14.0 @ 96 Scott Miller's lexus GS400
15.0 @ ??? Someone's volkswagon GTI
15.1 @ ??? Christian's boxter (with Tim driving)
Weight is the key for good ETs. The TR has considerably better power to weight ratio than the P4, but it can't effectively use most of the power until it gets into third gear. The viper is actually making more power than the F50, (Brian got a big kick out of that after his dyno run) but 350 lbs more than compensated for it.
I wanted to hit 140 in the TR, but the clutch started slipping on the last run and I called it a day.
I was actually surprised the F50 ran 122 mph, which is the same the F40 did on a 25 degree cooler day. I was running with the top off, so it might even be capable of going a bit faster with it on.
The F50 and the viper were both very consistant performers, but the TR and the supercharged M3 were all over the place with their runs.
Brian nocked over a tenth off of his times even in spite of the heat, due to launch practice and some inlet modifications. He also power shifted on his best run.
It was pretty funny watching the little volkswagon consistantly beat up on a tire shredding trans-am.
George Broussard had his newly hopped up 911 turbo, but it broke the trans on its very first run. We were expecting him to be in the 11's.
We probably won't run again until either I get the F50 souped up, or my GTO gets finished.
-----------------------------------------
John Carmack's .plan for May 22, 1998
-----------------------------------------
Congratulations to Epic, Unreal looks very good.
-----------------------------------------
John Carmack's .plan for Jun 08, 1998
-----------------------------------------
I spent quite a while investigating the limits of input under windows recently. I foudn out a few interesting things:
Mouse sampling on win95 only happens every 25ms. It doesn't matter if you check the cursor or use DirectInput, the values will only change 40 times a second.
This means that with normal checking, the mouse control will feel slightly stuttery whenever the framerate is over 20 fps, because on some frames you will be getting one input sample, and on other frames you will be getting two. The difference between two samples and three isn't very noticable, so it isn't much of an issue below 20 fps. Above 40 fps it is a HUGE issue, because the frames will be bobbing between one sample and zero samples.
I knew there were some sampling quantization issues early on, so I added the "m_filter 1" variable, but it really wasn't an optimal solution. It averaged together the samples collected at the last two frames, which worked out ok if the framerate stayed consistantly high and you were only averaging together one to three samples, but when the framerate dropped to 10 fps or so, you wound up averaging together a dozen more samples than were really needed, giving the "rubber stick" feel to the mouse control.
I now have three modes of mouse control:
in_mouse 1: Mouse control with standard win-32 cursor calls, just like Quake 2.
in_mouse 2: Mouse control using DirectInput to sample the mouse relative counters each frame. This behaves like winquake with -dinput. There isn't a lot of difference between this and 1, but you get a little more precision, and you never run into window clamping issues. If at some point in the future microsoft changes the implementation of DirectInput so that it processes all pending mouse events exactly when the getState call happens, this will be the ideal input mode.
in_mouse 3: Processes DirectInput mouse movement events, and filters the amount of movement over the next 25 milliseconds. This effectively adds about 12 ms of latency to the mouse control, but the movement is smooth and consistant at any variable frame rate. This will be the default for Quake 3, but some people may want the 12ms faster (but rougher) response time of mode 2.
It takes a pretty intense player to even notice the difference in most cases, but if you have a setup that can run a very consistant 30 fps you will probably apreciate the smoothness. At 60 fps, anyone can tell the difference, but rendering speeds will tend to cause a fair amount of jitter at those rates no matter what the mouse is doing.
DirectInput on WindowsNT does not log mouse events as they happen, but seems to just do a poll when called, so they can't be filtered properly.
Keyboard sampling appears to be millisecond precise on both OS, though.
In doing this testing, it has become a little bit more tempting to try to put in more leveling optimizations to allow 60 hz framerates on the highest end hardware, but I have always shied away from targeting very high framerates as a goal, because when you miss by a tiny little bit, the drop from 60 to 30 ( 1 to 2 vertical retraces ) fps is extremely noticable.
-
I have also concluded that the networking architecture for Quake 2 was just not the right thing. The interpolating 10 hz server made a lot of animation easier, which fit with the single player focus, but it just wasn't a good thing for internet play.
Quake 3 will have an all new entity communication mechanism that should be solidly better than any previous system. I have some new ideas that go well beyond the previous work that I did on QuakeWorld.
Its tempting to try to roll the new changes back into Quake 2, but a lot of them are pretty fundamental, and I'm sure we would bust a lot of important single player stuff while gutting the network code.
(Yes, we made some direction changes in Quake 3 since the original announcement when it was to be based on the Quake 2 game and networking with just a new graphics engine)
-----------------------------------------
John Carmack's .plan for Jun 16, 1998
-----------------------------------------
My last two .plan updates have described efforts that were not in our original plan for quake 3, which was "quake 2 game and network technology with a new graphics engine".
We changed our minds.
The new product is going to be called "Quake Arena", and will consist exclusively of deathmatch style gaming (including CTF and other derivatives). The single player game will just be a progression through a ranking ladder against bot AIs. We think that can still be made an enjoyable game, but it is definately a gamble.
In the past, we have always been designing two games at once, the single player game and the multi player game, and they often had conflicting goals. For instance, the client-server communications channel discouraged massive quantities of moving entities that would have been interesting in single player, while the maps and weapons designed for single player were not ideal for multiplayer. The largest conflict was just raw development time. Time spent on monsters is time not spent on player movement. Time spent on unit goals is time not spent on game rules.
There are many wonderful gaming experiences in single player FPS, but we are choosing to leave them behind to give us a purity of focus that will let us make significant advances in the multiplayer experience.
The emphasis will be on making every aspect as robust and high quality as possible, rather than trying to add every conceivable option anyone could want. We will not be trying to take the place of every mod ever produced, but we hope to satisfy a large part of the network gaming audience with the out of box experience.
There is a definite effect on graphics technology decisions. Much of the positive feedback in a single player FPS is the presentation of rich visual scenes, which are often at the expense of framerate. A multiplayer level still needs to make a good first impression, but after you have seen it a hundred times, the speed of the game is more important. This means that there are many aggressive graphics technologies that I will not pursue because they are not apropriate to the type of game we are creating.
The graphics engine will still be OpenGL only, with significant new features not seen anywhere before, but it will also have fallback modes to render at roughly Quake-2 quality and speed.
-----------------------------------------
John Carmack's .plan for Jul 04, 1998
-----------------------------------------
Here is the real story on the movement physics changes.
Zoid changed the movement code in a way that he felt improved gameplay in the 3.15 release.
We don't directly supervise most of the work Zoid does. One of the main reasons we work with him is that I respect his judgment, and I feel that his work benefits the community quite a bit with almost no effort on my part. If I had to code review every change he made, it wouldn't be worth the effort.
Zoid has "ownership" of the Quake, Glquake, and QuakeWorld codebases. We don't intend to do any more modifications at Id on those sources, so he has pretty free reign within his discretion.
We passed over the Quake 2 codebase to him for the addition of new features like auto download, but it might have been a bit premature, because official mission packs were still in development, and unlike glquake and quakeworld, Q2 is a product that must remain official and supported, so the scope of his freedoms should have been spelled out a little more clearly.
The air movement code wasn't a good thing to change in Quake 2, because the codebase still had to support all the commercial single player levels, and subtle physics changes can have lots of unintended effects.
QuakeWorld didn't support single player maps, so it was a fine place to experiment with physics changes.
QuakeArena is starting with fresh new data, so it is also a good place to experiment with physics changes.
Quake 2 cannot be allowed to evolve in a way that detracts from the commercial single player levels.
The old style movement should not be referred to as "real world physics". None of the quake physics are remotely close to real world physics, so I don't think one way is significantly more "real" than the other. In Q2, you accelerate from 0 to 27 mph in 1/30 of a second, which just as unrealistic as being able to accelerate in midair..
-----------------------------------------
John Carmack's .plan for Jul 05, 1998
-----------------------------------------
I am not opposed to adding a flag to control the movement styles. I was rather expecting it to be made optional in 3.17, but I haven't been directly involved in the last few releases.
The way this played out in public is a bit unfortunate. Everyone at Id is busy full time with the new product, so we just weren't paying enough attention to the Quake 2 modifications. Some people managed to read into my last update that we were blaming Zoid for things. Uh, no. I think he was acting within his charter (catering to the community) very well, it just interfered with an aspect of the game that shouldn't have been modified. We just never made it explicitly clear that it shouldn't have been modified.
It is a bit amusing how after the QuakeArena anouncement, I got flamed by lots of people for abandoning single player play (even though we aren't, really) but after I say that Quake 2 can't forget that it is a single player game, I get flamed by a different set of people who think it is stupid to care about single player anymore when all "everyone" plays is multiplayer. The joy of having a wide audience that knows your email address.
-----------------------------------------
John Carmack's .plan for Jul 16, 1998
-----------------------------------------
I have spent the last two days working with Apple's Rhapsody DR2, and I like it a lot.
I was dissapointed with the original DR1 release. It was very slow and seemed to have added the worst elements of the mac experience (who the hell came up with that windowshade minimizing?) while taking away some of the strengths of NEXTSTEP.
Things are a whole lot better in the latest release. General speed is up, memory consumption is down, and the UI feels consistant and productive.
Its still not as fast as windows, and probably never will be, but I think the tradeoffs are valid.
There are so many things that are just fundamentally better in the rhapsody design than in windows: frameworks, the yellow box apis, fat binaries, buffered windows, strong multi user support, strong system / local seperation, netinfo, etc.
Right now, I think WindowsNT is the best place to do graphics development work, but if the 3D acceleration issue was properly addressed on rhapsody, I think that I could be happy using it as my primary development platform.
I ported the current Quake codebase to rhapsody to test out conix's beta OpenGL. The game isn't really playable with the software emulated OpenGL, but it functions properly, and it makes a fine dedicated server.
We are going to try to stay on top of the portability a little better for QA. Quake 2 slid a bit because we did the development on NT instead of NEXTSTEP, and that made the irix port a lot more of a hassle than the original glquake port.
I plan on using the rhapsody system as a dedicated server during development, and Brian will be using an Alpha-NT system for a lot of testing, which should give us pretty good coverage of the portability issues.
I'm supposed to go out and have a bunch of meetings at apple next month to cover games, graphics, and hardware. Various parts of apple have scheduled meetings with me on three seperate occasions over the past couple years, but they have always been canceled for one reason or another (they laid off the people I was going to meet with once..).
I have said some negative things about MacOs before, but my knowledge of the mac is five years old. There was certainly the possibility that things had improved since then, so I spent some time browsing mac documentation recently. I was pretty amused. A stack sniffer. Patching trap vectors. Cooperative multitasking. Application memory partitions. Heh.
I'm scared of MacOS X. As far as I can tell, The basic plan is to take rhapsody and bolt all the MacOS APIs into the kernel. I understand that that may well be a sensible biz direction, but I fear it.
In other operating system news, Be has glquake running hardware accelerated on their upcoming OpenGL driver architecture. I gave them access to the glquake and quake 2 codebases for development purposes, and I expect we will work out an agreement for distribution of the ports.
Any X server vendors working on hardware accelerated OpenGL should get in touch with Zoid about interfacing and tuning with the Id OpenGL games on linux.
-----------------------------------------
John Carmack's .plan for Jul 29, 1998
-----------------------------------------
My F50 took some twin turbo vitamins.
Rear wheel numbers: 602 HP @ 8200 rpm 418 ft-lb @ 7200 rpm
This is very low boost, but I got the 50% power increase I was looking for, and hopefully it won't be making any contributions to my piston graveyard.
There will be an article in Turbo magazine about it, and several other car magazines want to test it out. They usually start out with "He did WHAT to an F50???" :)
Brian is getting a nitrous kit installed in his viper, and Cash just got his suspension beefed up, so we will be off to the dragstrip again next month to sort everything out again.
-----------------------------------------
John Carmack's .plan for Aug 17, 1998
-----------------------------------------
I added support for HDTV style wide screen displays in QuakeArena, so 24" and 28" monitors can now cover the entire screen with game graphics.
On a normal 4:3 aspect ratio screen, a 90 degree horizontal field of view gives a 75 degree vertical field of view. If you keep the vertical fov constant and run on a wide screen, you get a 106 degree horizontal fov.
Because we specify fov with the horizontal measurement, you need to change fov when going into or out of a wide screen mode. I am considering changing fov to be the vertical measurement, but it would probably cause a lot of confusion if "fov 90" becomes a big fisheye.
Many video card drivers are supporting the ultra high res settings like 1920 * 1080, but hopefully they will also add support for lower settings that can be good for games, like 856 * 480.
I spent a day out at apple last week going over technical issues.
I'm feeling a lot better about MacOS X. Almost everything I like about rhapsody will be there, plus some solid additions.
I presented the OpenGL case directly to Steve Jobs as strongly as possible.
If Apple embraces OpenGL, I will be strongly behind them. I like OpenGL more than I dislike MacOS. :)
-
Last friday I got a phone call: "want to make some exhibition runs at the import / domestic drag wars this sunday?". It wasn't particularly good timing, because the TR had a slipping clutch and the F50 still hasn't gotten its computer mapping sorted out, but we got everything functional in time.
The tech inspector said that my cars weren't allowed to run in the 11s at the event because they didn't have roll cages, so I was supposed to go easy.
The TR wasn't running its best, only doing low 130 mph runs. The F50 was making its first sorting out passes at the event, but it was doing ok. My last pass was an 11.8(oops) @ 128, but we still have a ways to go to get the best times out of it.
I'm getting some racing tires on the F50 before I go back. It sucked watching a tiny honda race car jump ahead of me off the line. :)
I think ESPN took some footage at the event.
-----------------------------------------
John Carmack's .plan for Sep 08, 1998
-----------------------------------------
I just got a production TNT board installed in my Dolch today.
The riva-128 was a troublesome part. It scored well on benchmarks, but it had some pretty broken aspects to it, and I never recommended it (you are better off with an intel I740).
There aren't any troublesome aspects to TNT. Its just great. Good work, Nvidia.
In terms of raw speed, a 16 bit color multitexture app (like quake / quake 2) should still run a bit faster on a voodoo2, and an SLI voodoo2 should be faster for all 16 bit color rendering, but TNT has a lot of other things going for it:
32 bit color and 24 bit z buffers. They cost speed, but it is usually a better quality tradeoff to go one resolution lower but with twice the color depth.
More flexible multitexture combine modes. Voodoo can use its multitexture for diffuse lightmaps, but not for the specular lightmaps we offer in QuakeArena. If you want shiny surfaces, voodoo winds up leaving half of its texturing power unused (you can still run with diffuse lightmaps for max speed).
Stencil buffers. There aren't any apps that use it yet, but stencil allows you to do a lot of neat tricks.
More texture memory. Even more than it seems (16 vs 8 or 12), because all of the TNT's memory can be used without restrictions. Texture swapping is the voodoo's biggest problem.
3D in desktop applications. There is enough memory that you don't have to worry about window and desktop size limits, even at 1280*1024 true color resolution.
Better OpenGL ICD. 3dfx will probably do something about that, though.
This is the shape of 3D boards to come. Professional graphics level rendering quality with great performance at a consumer price.
We will be releasing preliminary QuakeArena benchmarks on all the new boards in a few weeks. Quake 2 is still a very good benchmark for moderate polygon counts, so our test scenes for QA involve very high polygon counts, which stresses driver quality a lot more. There are a few surprises in the current timings..
-
A few of us took a couple days off in vegas this weekend. After about ten hours at the tables over friday and saturday, I got a tap on the shoulder..
Three men in dark suits introduced themselves and explained that I was welcome to play any other game in the casino, but I am not allowed to play blackjack anymore.
Ah well, I guess my blackjack days are over. I was actually down a bit for the day when they booted me, but I made +$32k over five trips to vegas in the past two years or so.
I knew I would get kicked out sooner or later, because I don't play "safely". I sit at the same table for several hours, and I range my bets around 10 to 1.
-----------------------------------------
John Carmack's .plan for Sep 10, 1998
-----------------------------------------
I recently set out to start implementing the dual-processor acceleration for QA, which I have been planning for a while. The idea is to have one processor doing all the game processing, database traversal, and lighting, while the other processor does absolutely nothing but issue OpenGL calls.
This effectively treats the second processor as a dedicated geometry accelerator for the 3D card. This can only improve performance if the card isn't the bottleneck, but voodoo2 and TNT cards aren't hitting their limits at 640*480 on even very fast processors right now.
For single player games where there is a lot of cpu time spent running the server, there could conceivably be up to an 80% speed improvement, but for network games and timedemos a more realistic goal is a 40% or so speed increase. I will be very satisfied if I can makes a dual pentium-pro 200 system perform like a pII-300.
I started on the specialized code in the renderer, but it struck me that it might be possible to implement SMP acceleration with a generic OpenGL driver, which would allow Quake2 / sin / halflife to take advantage of it well before QuakeArena ships.
It took a day of hacking to get the basic framework set up: an smpgl.dll that spawns another thread that loads the original oepngl32.dll or 3dfxgl.dll, and watches a work que for all the functions to call.
I get it basically working, then start doing some timings. Its 20% slower than the single processor version.
I go in and optimize all the queing and working functions, tune the communications facilities, check for SMP cache collisions, etc.
After a day of optimizing, I finally squeak out some performance gains on my tests, but they aren't very impressive: 3% to 15% on one test scene, but still slower on the another one.
This was fairly depressing. I had always been able to get pretty much linear speedups out of the multithreaded utilities I wrote, even up to sixteen processors. The difference is that the utilities just split up the work ahead of time, then don't talk to each other until they are done, while here the two threads work in a high bandwidth producer / consumer relationship.
I finally got around to timing the actual communication overhead, and I was appalled: it was taking 12 msec to fill the que, and 17 msec to read it out on a single frame, even with nothing else going on. I'm surprised things got faster at all with that much overhead.
The test scene I was using created about 1.5 megs of data to relay all the function calls and vertex data for a frame. That data had to go to main memory from one processor, then back out of main memory to the other. Admitedly, it is a bitch of a scene, but that is where you want the acceleration..
The write times could be made over twice as fast if I could turn on the PII's write combining feature on a range of memory, but the reads (which were the gating factor) can't really be helped much.
Streaming large amounts of data to and from main memory can be really grim. The next write may force a cache writeback to make room for it, then the read from memory to fill the cacheline (even if you are going to write over the entire thing), then eventually the writeback from the cache to main memory where you wanted it in the first place. You also tend to eat one more read when your program wants to use the original data that got evicted at the start.
What is really needed for this type of interface is a streaming read cache protocol that performs similarly to the write combining: three dedicated cachelines that let you read or write from a range without evicting other things from the cache, and automatically prefetching the next cacheline as you read.
Intel's write combining modes work great, but they can't be set directly from user mode. All drivers that fill DMA buffers (like OpenGL ICDs..) should definately be using them, though.
Prefetch instructions can help with the stalls, but they still don't prevent all the wasted cache evictions.
It might be possible to avoid main memory alltogether by arranging things so that the sending processor ping-pongs between buffers that fit in L2, but I'm not sure if a cache coherent read on PIIs just goes from one L2 to the other, or if it becomes a forced memory transaction (or worse, two memory transactions). It would also limit the maximum amount of overlap in some situations. You would also get cache invalidation bus traffic.
I could probably trim 30% of my data by going to a byte level encoding of all the function calls, instead of the explicit function pointer / parameter count / all-parms-are-32-bits that I have now, but half of the data is just raw vertex data, which isn't going to shrink unless I did evil things like quantize floats to shorts.
Too much effort for what looks like a reletively minor speedup. I'm giving up on this aproach, and going back to explicit threading in the renderer so I can make most of the communicated data implicit.
Oh well. It was amusing work, and I learned a few things along the way.
-----------------------------------------
John Carmack's .plan for Oct 14, 1998
-----------------------------------------
It has been difficult to write .plan updates lately. Every time I start writing something, I realize that I'm not going to be able to cover it satisfactorily in the time I can spend on it. I have found that terse little comments either get misinterpreted, or I get deluged by email from people wanting me to expand upon it.
I wanted to do a .plan about my evolving thoughts on code quality and lessons learned through quake and quake 2, but in the interest of actually completing an update, I decided to focus on one change that was intended to just clean things up, but had a surprising number of positive side effects.
Since DOOM, our games have been defined with portability in mind. Porting to a new platform involves having a way to display output, and having the platform tell you about the various relevant inputs. There are four principle inputs to a game: keystrokes, mouse moves, network packets, and time. (If you don't consider time an input value, think about it until you do - it is an important concept)
These inputs were taken in separate places, as seemed logical at the time. A function named Sys_SendKeyEvents() was called once a frame that would rummage through whatever it needed to on a system level, and call back into game functions like Key_Event( key, down ) and IN_MouseMoved( dx, dy ). The network system dropped into system specific code to check for the arrival of packets. Calls to Sys_Milliseconds() were littered all over the code for various reasons.
I felt that I had slipped a bit on the portability front with Q2 because I had been developing natively on windows NT instead of cross developing from NEXTSTEP, so I was reevaluating all of the system interfaces for Q3.
I settled on combining all forms of input into a single system event queue, similar to the windows message queue. My original intention was to just rigorously define where certain functions were called and cut down the number of required system entry points, but it turned out to have much stronger benefits.
With all events coming through one point (The return values from system calls, including the filesystem contents, are "hidden" inputs that I make no attempt at capturing, ), it was easy to set up a journalling system that recorded everything the game received. This is very different than demo recording, which just simulates a network level connection and lets time move at its own rate. Realtime applications have a number of unique development difficulties because of the interaction of time with inputs and outputs.
Transient flaw debugging. If a bug can be reproduced, it can be fixed. The nasty bugs are the ones that only happen every once in a while after playing randomly, like occasionally getting stuck on a corner. Often when you break in and investigate it, you find that something important happened the frame before the event, and you have no way of backing up. Even worse are realtime smoothness issues - was that jerk of his arm a bad animation frame, a network interpolation error, or my imagination?
Accurate profiling. Using an intrusive profiler on Q2 doesn't give accurate results because of the realtime nature of the simulation. If the program is running half as fast as normal due to the instrumentation, it has to do twice as much server simulation as it would if it wasn't instrumented, which also goes slower, which compounds the problem. Aggressive instrumentation can slow it down to the point of being completely unplayable.
Realistic bounds checker runs. Bounds checker is a great tool, but you just can't interact with a game built for final checking, its just waaaaay too slow. You can let a demo loop play back overnight, but that doesn't exercise any of the server or networking code.
The key point: Journaling of time along with other inputs turns a realtime application into a batch process, with all the attendant benefits for quality control and debugging. These problems, and many more, just go away. With a full input trace, you can accurately restart the session and play back to any point (conditional breakpoint on a frame number), or let a session play back at an arbitrarily degraded speed, but cover exactly the same code paths..
I'm sure lots of people realize that immediately, but it only truly sunk in for me recently. In thinking back over the years, I can see myself feeling around the problem, implementing partial journaling of network packets, and included the "fixedtime" cvar to eliminate most timing reproducibility issues, but I never hit on the proper global solution. I had always associated journaling with turning an interactive application into a batch application, but I never considered the small modification necessary to make it applicable to a realtime application.
In fact, I was probably blinded to the obvious because of one of my very first successes: one of the important technical achievements of Commander Keen 1 was that, unlike most games of the day, it adapted its play rate based on the frame speed (remember all those old games that got unplayable when you got a faster computer?). I had just resigned myself to the non-deterministic timing of frames that resulted from adaptive simulation rates, and that probably influenced my perspective on it all the way until this project.
Its nice to see a problem clearly in its entirety for the first time, and know exactly how to address it.
-----------------------------------------
John Carmack's .plan for Nov 03, 1998
-----------------------------------------
This was the most significant thing I talked about at The Frag, so here it is for everyone else.
The way the QA game architecture has been developed so far has been as two seperate binary dll's: one for the server side game logic, and one for the client side presentation logic.
While it was easiest to begin development like that, there are two crucial problems with shipping the game that way: security and portability.
It's one thing to ask the people who run dedicated servers to make informed decisions about the safety of a given mod, but its a completely different matter to auto-download a binary image to a first time user connecting to a server they found.
The quake 2 server crashing attacks have certainly proven that there are hackers that enjoy attacking games, and shipping around binary code would be a very tempting opening for them to do some very nasty things.
With quake and Quake 2, all game modifications were strictly server side, so any port of the game could connect to any server without problems. With Quake 2's binary server dll's not all ports could necessarily run a server, but they could all play.
With significant chunks of code now running on the client side, if we stuck with binary dll's then the less popular systems would find that they could not connect to new servers because the mod code hadn't been ported. I considered having things set up in such a way that client game dll's could be sort of forwards-compatable, where they could always connect and play, but new commands and entity types just might now show up. We could also GPL the game code to force mod authors to release source with the binaries, but that would still be inconvenient to deal with all the porting.
Related both issues is client side cheating. Certain cheats are easy to do if you can hack the code, so the server will need to verify which code the client is running. With multiple ported versions, it wouldn't be possible to do any binary verification.
If we were willing to wed ourselves completely to the windows platform, we might have pushed ahead with some attempt at binary verification of dlls, but I ruled that option out. I want QuakeArena running on every platform that has hardware accelerated OpenGL and an internet connection.
The only real solution to these problems is to use an interpreted language like Quake 1 did. I have reached the conclusion that the benefits of a standard language outweigh the benefits of a custom language for our purposes. I would not go back and extend QC, because that stretches the effort from simply system and interpreter design to include language design, and there is already plenty to do.
I had been working under the assumption that Java was the right way to go, but recently I reached a better conclusion.
The programming language for QuakeArena mods is interpreted ANSI C. (well, I am dropping the double data type, but otherwise it should be pretty conformant)
The game will have an interpreter for a virtual RISC-like CPU. This should have a minor speed benefit over a byte-coded, stack based java interpreter. Loads and stores are confined to a preset block of memory, and access to all external system facilities is done with system traps to the main game code, so it is completely secure.
The tools necessary for building mods will all be freely available: a modified version of LCC and a new program called q3asm. LCC is a wonderful project - a cross platform, cross compiling ANSI C compiler done in under 20K lines of code. Anyone interested in compilers should pick up a copy of "A retargetable C compiler: design and implementation" by Fraser and Hanson.
You can't link against any libraries, so every function must be resolved. Things like strcmp, memcpy, rand, etc. must all be implemented directly. I have code for all the ones I use, but some people may have to modify their coding styles or provide implementations for other functions.
It is a fair amount of work to restructure all the interfaces to not share pointers between the system and the games, but it is a whole lot easier than porting everything to a new language. The client game code is about 10k lines, and the server game code is about 20k lines.
The drawback is performance. It will probably perform somewhat like QC. Most of the heavy lifting is still done in the builtin functions for path tracing and world sampling, but you could still hurt yourself by looping over tons of objects every frame. Yes, this does mean more load on servers, but I am making some improvements in other parts that I hope will balance things to about the way Q2 was on previous generation hardware.
There is also the amusing avenue of writing hand tuned virtual assembly assembly language for critical functions..
I think this is The Right Thing.
-----------------------------------------
John Carmack's .plan for Nov 04, 1998
-----------------------------------------
More extensive comments on the interpreted-C decision later, but a quick note: the plan is to still allow binary dll loading so debuggers can be used, but it should be interchangable with the interpreted code. Client modules can only be debugged if the server is set to allow cheating, but it would be possible to just use the binary interface for server modules if you wanted to sacrifice portability. Most mods will be able to be implemented with just the interpreter, but some mods that want to do extensive file access or out of band network communications could still be implemented just as they are in Q2. I will not endorse any use of binary client modules, though.
-----------------------------------------
John Carmack's .plan for Dec 29, 1998
-----------------------------------------
I am considering taking a shortcut with my virtual machine implementation that would make the integration a bit easier, but I'm not sure that it doesn't compromise the integrity of the base system.
I am considering allowing the interpreted code to live in the global address space, instead of a private 0 based address space of its own. Store instructions from the VM would be confined to the interpreter's address space, but loads could access any structures.
On the positive side:
This would allow full speed (well, full interpreted speed) access to variables shared between the main code and the interpreted modules. This allows system calls to return pointers, instead of filling in allocated space in the interpreter's address space.
For most things, this is just a convenience that will cut some development time. Most of the shared accesses could be recast as "get" system calls, and it is certainly arguable that that would be a more robust programming style.
The most prevelent change this would prevent is all the cvar_t uses. Things could stay in the same style as Q2, where cvar accesses are free and transparantly updated. If the interpreter lives only in its own address space, then cvar access would have to be like Q1, where looking up a variable is a potentially time consuming operation, and you wind up adding lots of little cvar caches that are updated every from or restart.
On the negative side:
A client game module with a bug could cause a bus error, which would not be possible with a pure local address space interpreter.
I can't think of any exploitable security problems that read only access to the entire address space opens, but if anyone thinks of something, let me know.
-----------------------------------------
John Carmack's .plan for Dec 30, 1998
-----------------------------------------
I got several vague comments about being able to read "stuff" from shared memory, but no concrete examples of security problems.
However, Gregory Maxwell pointed out that it wouldn't work cross platform with 64 bit pointer environments like linux alpha. That is a killer, so I will be forced to do everything the hard way. Its probably for the best, from a design standpoint anyway, but it will take a little more effort.
-----------------------------------------
John Carmack's .plan for Jan 10, 1999
-----------------------------------------
Ok, many of you have probably heard that I spoke at the macworld keynote on tuesday. Some information is probably going to get distorted in the spinning and retelling, so here is an info dump straight from me:
Q3test, and later the full commercial Quake3: Arena, will be simultaniously released on windows, mac, and linux platforms.
I think Apple is doing a lot of things right. A lot of what they are doing now is catch-up to wintel, but if they can keep it up for the next year, they may start making a really significant impact.
I still can't give the mac an enthusiastic recommendation for sophisticated users right now because of the operating system issues, but they are working towards correcting that with MacOS X.
The scoop on the new G3 mac hardware:
Basically, its a great system, but Apple has oversold its performance reletive to intel systems. In terms of timedemo scores, the new G3 systems should be near the head of the pack, but there will be intel systems outperforming them to some degree. The mac has not instantly become a "better" platform for games than wintel, it has just made a giant leap from the back of the pack to near the front.
I wish Apple would stop quoting "Bytemarks". I need to actually look at the contents of that benchmark and see how it can be so misleading. It is pretty funny listening to mac evangelist types try to say that an iMac is faster than a pentium II-400. Nope. Not even close.
From all of my tests and experiments, the new mac systems are basically as fast as the latest pentium II systems for general cpu and memory performance. This is plenty good, but it doesn't make the intel processors look like slugs.
Sure, an in-cache, single precision, multiply-accumulate loop could run twice as fast as a pentium II of the same clock rate, but conversly, a double precision add loop would run twice as fast on the pentium II.
Spec95 is a set of valid benchmarks in my opinion, and I doubt the PPC systems significantly (if at all) outperform the intel systems.
The IO system gets mixed marks. The 66 mhz video slot is a good step up from 33 mhz pci in previous products, but that's still half the bandwidth of AGP 2X, and it can't texture from main memory. This will have a small effect on 3D gaming, but not enough to push it out of its class.
The 64 bit pci slots are a good thing for network and storage cards, but the memory controller doesn't come close to getting peak utilization out of it. Better than normal pci, though.
The video card is almost exactly what you will be able to get on the pc side: a 16 mb rage-128. Running on a 66mhz pci bus, it's theoretical peak performance will be midway between the pci and agp models on pc systems for command traffic limited scenes. Note that current games are not actually command traffic limited, so the effect will be significantly smaller. The fill rates will be identical.
The early systems are running the card at 75 mhz, which does put it at a slight disadvantage to the TNT, but faster versions are expected later. As far as I can tell, the rage-128 is as perfect as the TNT feature-wise. The 32 mb option is a feature ATI can hold over TNT.
Firewire is cool.
Its a simple thing, but the aspect of the new G3 systems that struck me the most was the new case design. Not the flashy plastic exterior, but the functional structure of it. The side of the system just pops open, even with the power on, and lays the motherboard and cards down flat while the disks and power supply stay in the encloser. It really is a great design, and the benefits were driven home yesterday when I had to scavenge some ram out of old wintel systems yesterday - most case designs suck really bad.
-
I could gripe a bit about the story of our (lack of) involvement with Apple over the last four years or so, but I'm calling that water under the bridge now.
After all the past fiascos, I had been basically planning on ignoring Apple until MacOS X (rhapsody) shipped, which would then turn it into a platform that I was actually interested in.
Recently, Apple made a strategic corporate decision that games were a fundamental part of a consumer oriented product line (duh). To help that out, Apple began an evaluation of what it needed to do to help game developers.
My first thought was "throw out MacOS", but they are already in the process of doing that, and its just not going to be completed overnight.
Apple has decent APIs for 2D graphics, input, sound, and networking, but they didn't have a solid 3D graphics strategy.
Rave was sort of ok. Underspecified and with no growth path, but sort of ok. Pursuing a proprietary api that wasn't competetive with other offerings would have been a Very Bad Idea. They could have tried to make it better, or even invent a brand new api, but Apple doesn't have much credebility in 3D programming.
For a while, it was looking like Apple might do something stupid, like license DirectX from microsoft and be put into a guaranteed trailing edge position behind wintel.
OpenGL was an obvious direction, but there were significant issues with the licensing and implementation that would need to be resolved.
I spent a day at apple with the various engineering teams and executives, laying out all the issues.
The first meeting didn't seem like it went all that well, and there wasn't a clear direction visible for the next two months. Finally, I got the all clear signal that OpenGL was a go and that apple would be getting both the sgi codebase and the conix codebease and team (the best possible arrangement).
So, I got a mac and started developing on it. My first weekend of effort had QuakeArena limping along while held together with duct tape, but weekend number two had it properly playable, and weekend number three had it brought up to full feature compatability. I still need to do some platform specific things with odd configurations like multi monitor and addon controlers, but basically now its just a matter of compiling on the mac to bring it up to date.
This was important to me, because I felt that Quake2 had slipped a bit in portability because it had been natively developed on windows. I like the discipline of simultanious portable development.
After 150 hours or so of concentrated mac development, I learned a lot about the platform.
CodeWarrior is pretty good. I was comfortable devloping there almost immediately. I would definately say VC++ 6.0 is a more powerful overall tool, but CW has some nice little aspects that I like. I am definately looking forward to CW for linux. Unix purists may be aghast, but I have allways liked gui dev environments more than a bunch of xterms running vi and gdb.
The hardware (even the previous generation stuff) is pretty good.
The OpenGL performance is pretty good. There is a lot of work underway to bring the OpenGL performance to the head of the pack, but the existing stuff works fine for development.
The low level operating systems SUCKS SO BAD it is hard to believe.
The first order problem is lack of memory management / protection.
It took me a while to figure out that the zen of mac development is "be at peace while rebooting". I rebooted my mac system more times the first weekend than I have rebooted all the WinNT systems I have ever owned. True, it has gotten better now that I know my way around a bit more, and the codebase is fully stable, but there is just no excuse for an operating system in this day and age to act like it doesn't have access to memory protection.
The first thing that bit me was the static memory allocation for the program. Modern operating systems just figure out how much memory you need, but because the mac was originally designed for systems without memory management, significant things have to be laid out ahead of time.
Porting a win32 game to the mac will probably involve more work dealing with memory than any other aspect. Graphics, sound, and networking have reasonable analogues, but you just can't rely on being able to malloc() whatever you want on the mac.
Sure, game developers can manage their own memory, but an operating system that has proper virtual memory will let you develop a lot faster.
The lack of memory protection is the worst aspect of mac development. You can just merrily write all over other programs, the development environment, and the operating system from any application.
I remember that. From dos 3.3 in 1990.
Guard pages will help catch simple overruns, but it won't do anything for all sorts of other problems.
The second order problem is lack of preemptive multitasking.
The general responsiveness while working with multiple apps is significantly worse than windows, and you often run into completely modal dialogs that don't let you do anything else at all.
A third order problem is that a lot of the interfaces are fairly clunky.
There are still many aspects of the mac that clearly show design decisions based on a 128k 68000 based machine. Wintel has grown a lot more than the mac platform did. It may have been because the intel architecture didn't evolve gracefully and that forced the software to reevaluate itself more fully, or it may just be that microsoft pushed harder.
Carbon sanitizes the worst of the crap, but it doesn't turn it into anything particularly good looking.
MacOS X nails all these problems, but thats still a ways away.
I did figure one thing out - I was always a little curious why the early BeOS advocates were so enthusiastic. Coming from a NEXTSTEP background, BeOS looked to me like a fairly interesting little system, but nothing special. To a mac developer, it must have looked like the promised land..
-----------------------------------------
John Carmack's .plan for Jan 29, 1999
-----------------------------------------
The issue of the 8192 unit map dimension limit had been nagging at me for a long time now. The reason for the limit is that coordinates are communicated over the network in 16 bit shorts divided as a sign bit, twelve unit bits, and three fractional bits. There was also another side to the representation problem, but it was rarely visible: if you timescale down, you can actually tell the fractional bit granularity in position and velocity.
The rest of the system (rendering and gameplay) has never had any issues with larger maps, even in quake 1. There are some single precision floating point issues that begin to creep in if things get really huge, but maps should be able to cover many times the current limit without any other changes.
A while ago I had changed Q3 so that the number of fractional bits was a compile time option, which allowed you to trade off fine grain precision for larger size. I was considering automatically optimizing this for each level based on its size, but it still didn't feel like a great solution.
Another aspect of the problem that wasn't visible to the public was that the the fractional quantization of position could cause the position to actually be inside a nearby solid when used for client side prediction. The code had to check for this and try to correct the situation by jittering the position in each of the possible directions it might have been truncated from. This is a potential issue whenever there is any loss of precision whatsoever in the server to client communication.
The obvious solution is to just send the full floating point value for positions, but I resisted that because the majority of our network traffic is positional updates, and I didn't want to bloat it. There have been other bandwidth savings in Q3, and LANs and LPB connections are also relevent, so I was constantly evaluating the tradeoff.
Dealing with four or five players in view isn't a real problem. The big bandwidth issues arrive when multiple players start unloading with rapid fire weapons. (as an aside, I tried making 5hz fire weapons for Q3 to save bandwidth, but no matter how much damage they did, 5hz fire rates just seemed to feel slow and weak...)
I finally moved to a bit-level stream encoding to save some more bandwidth and give me some more representational flexibility, and this got me thinking about the characteristics of the data that bother us.
In general, the floating point coordinates have significant bits all through the mantissa. Any movement along an angle will more or less randomize the low order bits.
My small little insight was that because missiles are evaluated parameterically instead of itteretively in Q3, a one-time snapping of the coordinates can be performed at their generation time, giving them fixed values with less significant bits for their lifetime without any effort outside their spawning function. It also works for doors and plats, which are also parametrically represented now. Most events will also have integral coordinates.
The float encoder can check for an integral value in a certain range and send that as a smaller number of bits, say 13 or so. If the value isn't integral, it will be transmitted as a full 32 bit float.
The other thing I am investigating is sub-byte delta encoding of floating point values. Even with arbitrary precision movement deltas, the sign and exponent bits change with very low frequency except when you are very near the origin. At the minimum, I should be able to cut the standard player coordinate delta reps to three bytes from four.
So, the bottom line is that the bandwidth won't move much (it might even go down if I cut the integral bits below 15), the maps become unbounded in size to the point of single precision roundoff, and the client doesn't have to care about position jittering (which was visible in Q3 code that will be released).
-----------------------------------------
John Carmack's .plan for Mar 03, 1999
-----------------------------------------
On the issue of railgun firing rates -- we played with it for a while at the slower speed, but it has been put back to exactly Q2's rate of fire.
I do agree with Thresh that the way we had it initially (faster than Q2, but with the same damage) made it an overpowered weapon in the hands of highly skilled players, which is exactly what we should try to avoid.
An ideal game should give scores as close to directly proportional to the players reletive skills as possible. The better player should win in almost all cases, but the game will be more entertaining if the inferior players are not completely dominated.
Quake 1 had really bad characteristics that way -- Thresh can play extremely talented players and often prevent them from scoring a single point. We wouldn't put up with a conventional sport that commonly game scores of 20 to 1 in championship matches, and I don't think we should encourage it in our games.
Eliminating health items is probably the clearest way to prevent blowout games, but that has never been popular. Still, we should try to avoid weapon decisions that allow the hyper-skilled to pull even farther away from the rest of the crowd. They will still win, no matter what the weapons are, just not by as wide a margin.
-----------------------------------------
John Carmack's .plan for Mar 17, 1999
-----------------------------------------
First impressions of the SGI visual workstation 320:
I placed an order for a loaded system ($11k) from their web site two months ago. It still hasn't arrived (bad impression), but SGI did bring a loaner system by for us to work with.
The system tower is better than standard pc fare, but I still think Apple's new G3 has the best designed computer case.
The wide aspect LCD panel is very nice. A while ago I had been using a dual monitor LCD setup on a previous intergraph, but the analog syncing LCD screens had some fringing problems, and the gamma range was fairly different from a CRT. The SGI display is perfectly crisp (digital interface), and has great color. The lighting is a little bit uneven top to bottom on this unit -- I am interested to see how the next unit looks for comparison.
Unfortunately, the card that they bundle with the LCD if you buy it separately is NOT a good 3D accelerator, so if you care about 3D and want this LCD screen, you need to buy an sgi visual workstation. Several of the next generation consumer cards are going to support digital flat panel outputs, so hopefully soon you will be able to run a TNT2 or something out to one of these.
The super memory system does not appear to have provided ANY benefit to the CPU. My memory benchmarking tests showed it running about the same as a standard intel design.
Our first graphics testing looked very grim -- Quake3 didn't draw the world at all. I spent a while trying to coax some output by disabling various things, but to no avail. We reported it to SGI, and they got us a fix the next day. Some bug with depthRange(). Even with the fix, 16 bit rendering doesn't seem to work. I expect they will address this.
Other than that, there haven't been any driver anomolies, and both the game and editor run flawlessly.
For single pass, top quality rendering (32 bit framebuffer, 32 bit depth buffer, 32 bit trilinear textures, high res screen), the SGI has a higher fill rate than any other card we have ever tested on a pc, but not by too wide of a margin.
If your application can take advantage of multitexture, a TNT or rage128 will deliver slightly greater fill performance. It is likely that the next speed bump of both chips will be just plain faster than the SGI on all fill modes.
A serious flaw is that the LCD display can't support ANY other resolutions except the native 1600*1024. The game chunks along at ten to fifteen fps at that resolution (but it looks cool!). They desperately need to support a pixel doubled 800*512 mode to make any kind of full screen work possible. I expect they will address this.
Vsync disable is implemented wrong. Disabling sync causes it to continue rendering, but the flip still doesn't happen until the next frame. This gives repeatable (and faster) benchmark numbers, but with a flashing screen that is unusable. The right way is to just cause the flip to happen on the next scan line, like several consumer cards do, or blit. It gives tearing artifacts, but it is still completely usable, and avoids temporal nyquist issues between 30 and 60 hz. I expect they will address this.
Total throughput for games is only fair, about like an intergraph. Any of the fast consumer cards will run a quake engine game faster than the sgi in its current form. I'm sure some effort will be made to improve this, but I doubt it will be a serious focus, unless some SGI engineers develop unhealthy quake addictions. :-)
The unified memory system allows nearly a gig of textures, and individual textures can by up to 4k by 4k. AGP texturing provides some of this benefit for consumer cards, but not to the same degree or level of performance.
The video stream support looks good, but I haven't tried using it yet.
Very high interpolater accuracy. All the consumer cards start to break up a bit with high magnification, weird aspects, or long edges. The professional cards (intergraph, glint, E&S, SGI) still do a better job.
SGI exports quite a few more useful OpenGL extensions than intergraph does, but multisample antialiasing (as claimed in their literature) doesn't seem to be one of them.
Overall, it looks pretty good, and I am probably going to move over to using the SGI workstation full time when my real system arrives.
I was very happy with the previous two generations of intergraph workstations, but this last batch (GT1) has been a bunch of lemons, and the wildcat graphics has been delayed too long. The current realizm-II systems just don't have enough fill rate for high end development.
For developers that don't have tons of money, the decision right now is an absolute no-brainer -- buy a TNT and stick it in a cheap system. Its a better "professional" 3D accelerator than you could buy at any price not too long ago.
-----------------------------------------
John Carmack's .plan for Apr 24, 1999
-----------------------------------------
We are finally closing in on the first release of Q3test.
As you have probably heard by now, the first release in going to be the mac version, probably followed by the linux version, and only then the windows version.
Some of you are busy getting all bent out of shape about this.
We want to get a lot of people playing with the test to find bugs and offer well thought out suggestions, but there are classes of bugs and suggestions that emerge in each order of magnitude of exposed testers.
If a given bug is going to show up when a thousand people have looked at it, but we had released it to a hundred thousand people, then we are going to have a lot of duplication to wade through.
The mac testers will find some obvious problems. We will fix them. The later releases will be better.
Even if we had the windows distribution ready to go right now, I would still seriously consider releasing one of the mac or linux versions first because it will make our tracking a lot easier.
The holdup on the windows side are the issues with updated driver distribution. The game itself doesn't have any holdups.
We could do a windows release for just, say, voodoo2 owners and get some of the benefit of a controlled release, but it wouldn't really work out, because everyone would figure out that it can be made to (almost) work on lots of other cards by tweaking some values. That type of feedback would not be useful, because we KNOW that there are problems with most of the current drivers. We have been working with all of the vendors for the past year to get them all straightened out.
Glsetup is going to be slick -- just run it and it will Do The Right Thing for your video configuration.
We hope it will be done soon, but there are factors out of our direct control involved.
Don't be spiteful. This is just the beginning of the testing and release process.
One conspiracy theory suggests that Apple is somehow getting us to do this.
What we have "gotten" from Apple is a few development machines. No cash payoff. No bundling deal. No marketing contract.
I am looking at this long term. I want to see OS X become a top notch platform for graphics development. I think highly of the NEXTSTEP heritage and I might move my development from NT if it turns out well. There is a lot of groundwork that needs to be laid with apple for this to happen, and my working on the mac right now is part of that. Plus a lot of complaining to various apple engineers and executives. :-)
To be clear:
At this time, there is no mac that is as fast for gaming (or just about anything, actually) as a pentium III with a top of the line 3D card. Period. I have been misquoted by some mac evangelists as saying otherwise.
The new (blue and white) G3 systems are very good systems in many ways, and make a perfectly good gaming platform. However, a high end wintel machine just has more horsepower on both the CPU and the 3D card.
A 400 mhz G3 performs about the same as a 400 mhz PII if they aren't fill rate limited, where the faster cards on the PC will give about a 25% advantage. A 500 mhz PIII with an appropriate card in 30% faster than the best mac you can buy.
The multi colored iMacs, old G3 desktops, and powerbooks can play Quake3, but the RagePro 3D acceleration defines the absolute bottom end of our supported platforms. A serious gamer will not be satisfied with it.
Voodoo cards are not currently supported by the OpenGL driver, which is very unfortunate because many serious mac gamers own voodoo cards. I hope Apple and 3dfx will be able to correct this soon, but I certainly understand Apple's prioritization -- obviously, good support for the OEM video options is of primary importance.
The voodoo performance will still lag the windows platform by some amount, but some strides have been made in that area recently, so I expect good performance,
Gaming is not a reason to buy a mac, but Apple is taking steps so that it may not be a reason to avoid a mac if you have other reasons for wanting one.
MacOS still sucks.
-----------------------------------------
John Carmack's .plan for Apr 25, 1999
-----------------------------------------
Some people seem to think that I just make up these performance comparison numbers. I don't. I measure things, and I understand control conditions.
In this discussion, assume "wintel" is a 500 mhz PIII with either a agp rage128, or an agp TNT card, and "macos" is a 400 mhz G3 with the pci rage128.
At the highest level, you can make application class comparisons between platforms. For instance, CodeWarrior on the mac compiles faster than VC++ on wintel, but stuffit is way slower than winzip. This is useful data, but says more about the application design than the relative merits of the platforms. CW uses a single object file repository, for instance.
A better comparison is an identical app on both platforms.
Photoshop is often faster on macos than wintel. There is certainly a lot of common code, but individual filters are optimized for each platform. Some of these hand optimized operations are significantly faster on the mac.
Quake1 was the counterpoint to that. Quake1 had significant amounts of hand tuned asm code for intel, and the PPC version never got as much attention. The PPC version was noticeably slower (you would have to time at 640*480 to avoid unfairly penalizing the mac for lack of low res modes).
So, clearly, hand tuned asm code can make either platform pull ahead. It also shows that the two platforms are at least fairly close in performance. I never said macs were SLOW, just not quite as fast as the best intel systems.
Quake3 doesn't software rasterize, so there isn't any great place for lots of asm code (the great place is in the OpenGL driver). The code is essentially identical on all platforms.
Q3 is definitely faster on a wintel system than a macos system. When the wintel version is released, everyone will be independantly repeating that measurement.
Even this measurement isn't exactly an apples to apples comparison, because the OpenGL driver and 3D card are still a significant variance. The two can be broken out farther: Q3 can be run without 3D output to test just the identical compiled code. Wintel is still faster, although somewhat less so. The OpenGL + 3D card setup can be benchmarked separately on the axis of throughput and fill rate, which show the intel system being significantly faster. I can't break that apart into the two separate components, but I will guess that the OpenGL driver is probably as efficient as the wintel drivers and the performance delta is due to the system interface and the video card. The current mac rage128 cards run at 75 mhz, which is a third slower than the PC cards. AGP is also more than just a faster PCI, it can influence the structure of communication with the card.
It has been my observation in the past that most of my code tracks just about midway between specint and specfp for performance comparisons. There is a lot of floating point, but it is all short vectors, rather than the massive vectors of scientific computing. If we discount the graphics subsystem, Q3 follows this reasonably well. The intel system does slightly better than projected.
"Sucks" is a subjective description that can be dismissed as opinion. Note that I have NEVER said that the hardware sucks, or the user interface sucks, just that the mac OPERATING SYSTEM sucks.
"Faster", when qualified with testing conditions, is objective, and all the wishing in the world doesn't change it.
Objectivity and quantification are the paths to improvement.
I will be very happy if Apple can produce a desktop system that is faster than anything else you can get. I respect good engineering from any source. Altivec should be better than the PIII extensions (trinary ops -- yeah!). The upcoming system architectures look good. They have a shot at it, but they won't make it if they complacently think "oh, we are already faster than any pc system".
My twin turbo F50 can still be outrun at the dragstrip by much cheaper race cars. Many ferrari owners would not dare set foot at a drag strip, because they fear objective measurements that may not show their important possession in the best light. I would rather have the facts, so I can base future decisions on logical grounds.
-----------------------------------------
John Carmack's .plan for Apr 26, 1999
-----------------------------------------
Interpreting the lagometer (the graph in the lower right corner):
The upper graph (blue/yellow) slides one pixel for every rendered frame. Blue lines below the baseline mean that the frame is interpolating between two valid snapshots. Yellow lines above the baseline mean the frame is extrapolating beyond the latest valid time. The length of the line is proportional to the time.
The lower graph (green/yellow/red) slides one pixel for every received snapshot. By default, snapshots come 20 times a second, so if you are running >20 fps, the top graph will move faster, and vice versa. A red bar means the snapshot was dropped by the network. Green and yellow bars are properly received snapshots, with the height of the bar proportional to the ping. A yellow bar indicates that the previous snapshot was intentionally supressed to stay under the rate limit.
The upper graph indicates the consistancy of your connection. Ideally, you should always have blue bars of only a pixel or two in height. If you are commonly getting big triangles of yellow on the graph, your connection is inconsistant.
In a heavy firefight, it is normal for modem players to see yellow bars in the bottom graph, which should return to green when the action quiets down. If you are getting several red bars visible, you may want to look for a server that drops less packets.
There are a few tuning variables for people trying to optimize their connection:
The most important one is "rate", which is what the connection speed option in the menu sets.
We are fairly conservative with the values we set for the given modem speeds: 2500 for 28.8, 3000 for 33, and 3500 for 56k.
You may actually be connecting faster than that, and modem compression may be buying you something, so you might get a better play experience by increasing the values slightly.
If you connect at 50000 bps, try a rate of 5000, etc.
I err on the conservative side, because too low of a rate will only make the movement of other things in the world choppy, while too high of a rate can cause huge amounts of lag.
Note that the optimal rate will be somewhat lower than a rate for QW or Q2, because I now include the UDP packet header length in the bandwidth estimate.
You can ask for a different number of snapshots by changing the "snaps" variable, but there isn't a lot of benefit to that. Dedicated servers run at 40hz, so stick to divisors of that: 40, 20 (default), 10. A snaps of 40 will usually just cause you to hit your rate limit a lot faster. It may be usefull for tuning rate, if nothing else.
You can adjust the local timing point with "cg_timenudge ", which effectively adds local lag to try to make sure you interpolate instead of extrapolate. If you really want to play on a server that is dropping a ton of packets, a timenudge of 100 or so might make the game smoother.
------
One more addition to net cvars:
"cl_maxpackets" will restrict the maximum number of outgoing packets to prevent client to server rate problems. This does not limit the client framerate. This defaults to 20, which might actually be a bit low. You might try experimenting with raising this to 40.
"cl_maxfps" still exists, but it will never need to be used for networking reasons.
-------
* converted cvar allocation to indexes to allow range checking
* cgame converted over to use vmCvar_t instead of cvar_t needed for interpreted cgame
* fixed server crashing string bug
* adjusted scoreboard for 8 players
* show hostname on connection screen
* fixed null model warning on startup
* more space for hostname on local servers screen
* fixed mac Open Transport memory buffer bug, this was causing most of the mac crashes
* made Info_ValueForKey() case insensitive
* sv_privateClients, sv_privatePassword. this allows you to reserve slots on a public server for password access while allowing most to be freely available
* "server is full" message on connect screen
* archive handicap in config file
* cheat protect r_nocurves
* byte order independent zip checksum
* removed cl_stereo, use glConfig.stereoEnabled
-----------------------------------------
John Carmack's .plan for Apr 27, 1999
-----------------------------------------
* cgame converted to use local buffer based lerpTag for interpretation
* cgame converted to use local buffer based argv for interpretation
* new sound code to remove latency
* added drop shadow to field characters and fixed scrolling
* fixed edge-of-bounce-pad misprediction error (server side)
* remove broken weapon-stay dmflag
* made menu gfx never picmip
* cheat protect r_lightmap
* clear sound buffer before any file IO
* use GetOSEvent instead of WaitNextEvent on mac when fullscreen. removes hitches caused by other tasks and gives a performance boost
* continuous scoreboard / ping update when tab is down
* put version number on menu background
* fixed toggle cvar bug
* dim out behind floating menus
-----------------------------------------
John Carmack's .plan for Apr 28, 1999
-----------------------------------------
* converted sound positioning away from callback method
* increased mac memory zone by 5 megs
* new memory allocator for temporary render use during init
* converted cmodel references to handles and range checked
* converted sound references to handles and range checked
* converted file references to handles and range checked
-----------------------------------------
John Carmack's .plan for Apr 29, 1999
-----------------------------------------
* rework versioning for architecture tracking
* use a seperate endpoint for address resolves on mac
* hide OTLook warnings if "developer" isn't set
* defered mac renderer scanning until after mode set so 8 bit desktops don't confuse it
* global motd/update server
* fixed view model animations on models with custom anims
technical note:
Q3 can run networked player movement in either an asynchronous or synchronous manner. The default mode is to allow all client movement to be done asynchronously with the servers advancement of time.
The primary reason is to allow player movement to be predicted on the client side. The primary drawback is that while your movement is smooth, the other players that you see running around in the world move with a jerkiness that is relative to their framerate and network connection quality. It is NOT necessarily relative to their ping - a player on a fast system with a clean modem connection can move smoothly. If you see a player stuttering around, either they have a bad framerate, or the network connection between them and the server or you and the server is poor. The amount of stuttering is sort of the sum of the dropped or variable packets on BOTH connections.
You can force Q3 to run all clients synchronously by setting "g_synchronousClients 1" on the server. This will make Q3 behave similar to Q1 for networking. All movement will be lagged except view angles, which are still short-circuited immediately.
Some people claim to prefer synchronous movement when everyone had a very good ping, but I don't personally think it is ever a play benefit. It makes leading players a bit easier, but I think the crisp movement control of client side prediction is a much better tradeoff.
However, there is still a reason for using it: recorded demos come out a LOT smoother looking when running with sync. Note that q3test does not allow demo recording and playback, so this is just for future reference...
-----------------------------------------
John Carmack's .plan for Apr 30, 1999
-----------------------------------------
I put together a document on optimizing OpenGL drivers for Q3 that should be helpfull to the various linux 3D teams.
http://www.quake3arena.com/news/glopt.html
-------
* vmtest framework, q3asm work
* converted scene building to procedural style. allows better error checking, better performance characteristics when interpreted, and is a setup stage for SMP optimizations in the renderer if I get around to it
* protected some potential div by 0 areas in cgame
-----------------------------------------
John Carmack's .plan for May 04, 1999
-----------------------------------------
* seeded random numbers differently on tourney restarts
* fixed events on initial snapshots
* removed g_maxentities configuration, set by G_ENTITY_BITS
* cl_motd 0 to allow never sending request packets
* fixed map cache clearing bug
* cg_drawFPS 1 for running fps counter in corner
* remove all teleport destination pads
* moved checkmap out of cgame
* moved time positioning out of cgame
* made usercmd overrun freeze in place instead of snapping back
* slightly increased shotgun spread
* protected against using a cleared clientinfo
* use snapped origin from players for linking to prevent slight prediction errors during player collisions
-----------------------------------------
John Carmack's .plan for May 05, 1999
-----------------------------------------
* client side predict item pickups. running over items was one of the few remaining locally perceived signs of lag
* new pont-in-patch test code
* fixed pathname errors when mac users had slashes in their paths: "B/W mac". sigh.
-----------------------------------------
John Carmack's .plan for May 07, 1999
-----------------------------------------
* changed grabbed items to SVF_NOCLIENT instead of EF_NODRAW now that the pickup event is on the player
* clear event flags with event on reset
* move playerstate to netfield bit communication
* fixed configstring delta sequencing issue after initial gamestate
* extended the netgraph: short red lines are missing client to server packets (need to drop 3 in a row)
* extended cg_debugevents
* increased cl_maxpackets to 30
* fixed bug with console field not getting drawwidth set
* fastsky implies noportals
* changed fastsky color
* q3map now fixes tjunctions at fog boundaries
* build optimized tree with visible hulls of sides
* adjusted plane culling to avoid some cracks
* r_facePlaneCull
* fixed too-lax colinear collapse to avoid some cracks
-----------------------------------------
John Carmack's .plan for May 08, 1999
-----------------------------------------
There is one must-fix issue and a couple smaller issues remaining before the release candidate build, then we have to do a lot of testing on it. I made a lot of significant changes in the last week, and I'm sure there are some things we still need to sort out before we inflict it on the general public.
We are aiming for sunday, but understand that that means sunday evening, not sunday morning.
If saturday night / sunday morning testing on the release candidate turns up significant problems, we will put off the release until they are fixed. That could be later sunday night, or it might not make it until monday night.
The previous release delays for win32 were issues out of our control, but this release rests squarely on me. The content and other issues are ready, but we still need to make sure all the new code is solid.
* fixed give item bug
* new first snapshot timing
* moved sun drawing outside of sky shader to fix showtris
* r_drawSun
* handle all shader tesselations in q3map with tjunc fixups
* different flatness epsilons for edge vs border grids
* reorganize sound directories
* removed footsteps on non-metalic and non-water surfaces
* fixed bug with multiple looping sounds
* client side predict teleporters, go to "hyperspace"
* precache remaining liquid sounds
* don't fire jumppad events if upward velocity
-----------------------------------------
John Carmack's .plan for May 09, 1999
-----------------------------------------
We would up making tweaks to both maps today, so the data didn't reach final form until a few hours ago.
I just finished making release candidates for all three architectures, but I already found a couple problems that need to be fixed.
If everything goes perfectly (ha), and I nail these problems immediately when I wake up, then we might make it out tonight, but it is looking a bit doubtful.
There are a few known issues that I decided NOT to hold the test up for:
The gauntlet is functioning correctly, but the visuals are wrong. The designed behavior is that when you hold down attack it will scan for a target and only punch forward when it hits. The visuals currently show it punching constantly.
Dynamic lighting is currently taking a really excessive amount of cpu time. If you are having performance problems in firefights, you may want to turn it off. The option is in the preferences, or you can just issue "r_dynamicLighting 0" at the console.
The powerup item sounds aren't global across the entire world since I went to the client side predicted items.
There are some cases when a weapon that was picked up with a predicted item and immediately fired doesn't make a muzzle flash.
* fixed fs_copyfiles after ospath split
* fixed look-at-killer
* changed railgun impact to plasma dish
* convert connect packet to infostring
* put footsteps back in...
* r_drawsun 0 by default to avoid probs for now
* fixed event clear overwrite problem
* client side predict weapon switch on item pickup
* changed sound fallbacks to "visor" from "male"
* made turbulent texcoords based off of xyz instead of st
-----------------------------------------
John Carmack's .plan for May 10, 1999
-----------------------------------------
A good day of work. I just finished a long test game with all three architectures, and everything looks solid.
As far as I can tell, these are ready to go after making installers and such, but everyone else has an oportunity to find bugs while I sleep...
We won't hold up for minor gameplay issues, but if anyone turns up a repeatable crasher I will rebuild everything.
Barring problems, we should start rolling the releases out tonight.
* fixed crash case on fallback from an unsupported fullscreen
* fixed overrun with very fast system connecting to a very lagged server
* fixed bad Z_Free on sounds not found
* fixed autoswitch with sync clients
* fixed losing console field on positive histories
* fixed demo recording and playback with new net code
* handle signed bit fields in msg code
* fixed playerstate event bit loss on encoding
* fixed "bad clientnum on player entity"
* reenabled corpses sinking into ground
-----------------------------------------
John Carmack's .plan for May 11, 1999
-----------------------------------------
I am offering a bounty for server crashing bugs. Q2 had several releases forced out because of malicious attacks on all the public servers, so I want to try and flush out what I can during Q3's testing phase.
There is a server running in the debugger here at crashtest.idsoftware.com (192.246.40.68). Anyone that can repeatably hang or crash this system can have a $100 prize and some misc bit of Q3A paraphenalia that I can dig up.
Operating system level attacks don't count -- only things that I can actually fix or protect against in my code.
Denial of service attacks don't count if they require upkeep, but if there is a fire-and-forget DOS attack, it will still count.
Any actions you can perform with the released client are fair game. Crashing the client isn't good for a bounty, but I would still like to know about it.
Custom attack programs are also fair game. These are actually what I am most concerned about -- malicious programs that goes through and crash all listed servers.
Ideally, you would practice on a private server under your control and only hit crashtest when you think you can repeat it.
If you find one, email me the instructions so I can reproduce it. Include "CRASHTEST" in the subject so I won't miss it.
First come, first served, one bounty per bug. I will update crashtest with our internal builds, so it will certainly be possible that an attack on the released servers no longer functions on crashtest.
-------------
Now that the first win32 test is out, here is The Plan for going forward:
All future releases should be same-day for all architectures.
There may be an exe-only update to the current distributions if there are significant problems, but it isn't scheduled.
The next major test release will include a new one on one map designed for tournement play, and new executables with server and game modifications, but will not require downloading a new pak0.pk3.
The release after that will introduce various teamplay rules on the original two maps. This version will likely be another full download, because I know that I still have a couple things to change in the map format. This will probably be the first test running with the virtual machine.
The final major test release will introduce the single player game with bots and ranks.
After any bugs are shaken out of that, it will be the "Q3 Demo" instead of the "Q3 Test", and we should be ready to release the full game to stores.
In an ideal world, people that aren't prepared to deal with in-development software would wait until then to form an opinion of the product.
-------
Everyone should realize that many popular net links are going to be clogged up with q3test downloads for a while, so net play may be a bit patchy to a lot of servers.
-------
BigImp wins the first prize. It doesn't crash the server, but fmtspec names will crash all clients that try to log on. Technically that would be an upkeep required DOS attack, but I'll let this one go.
I even had a "FIXME: make vsprintf safe" comment by the offending line...
I am going to update the server to filter out all % chars that come in over the net to prevent any other similar things.
------
Sami Tammilehto wins the second prize. Some large connectionless packets can cause crashes.
This one was a result of me having the maximum token size defined lower than the maximum string size.
-------
Do NOT send bug reports and game comments directly to me! If I have to filter through hundreds of emails a day, I won't get any more work done... Only crashtest related problems should come to me, everything else should go to q3feedback@idsoftware.com.
-------
You can bias the level of detail lower than allowed in the menu with "r_lodbias 2", which will force all models to the lowest lod. The view weapon will look very ugly.
Another little speedup option that isn't offered in the menus is: "cg_simpleitems 1" this removes the extra rings and spheres around some items.
You can also turn off all the gibs with "cg_gibs 0".
* clear game memory at init, which fixes the stuck-at-intermission problem on mac servers
* fixed mismatched free / Z_Free in demo menu
* removed unused reference to sprites/plama.md3
* automatically get sounds from model name
* scale sensitivity by zoom
* immediately archive changes to latched cvars
* cheat protect r_portalonly
* don't print "XXX connected" on level restarts
* fixed "give item" on levels where 0,0,0 is in solid
* fixed timedemo
* don't play pain falling sound if dead
* fixed falling damage sound not snd specific
* fixed crashtest 2
* fixed crashtest 1
* q3map_backshader
* q3map_globaltexture
-----------------------------------------
John Carmack's .plan for May 12, 1999
-----------------------------------------
We had to upgrade the crashtest machine to NT sp 5, because some people were attacking it with windows crashers. Those don't count.
Crashtest #3 from [iBO]QWhAX0R was a combination of two problems:
The symptom was disconnecting all clients with an illegible server message. This turned out to be caused by the fact that I was parsing strings out of my net buffers with a MSG_ReadChar() function, and I was checking for EOF as a negative one return value. I had to change this to a MSG_ReadByte() call, because -1 was showing up in the messages, which then caused a parse error because it wasn't really the end of the message.
The actual root of that issue was code like this:
[code]
{
char buffer[MAX_STRING_CHARS];
...
strncpy( buffer, input, sizeof(buffer) - 1 );
...
}
[/code]
No buffer overruns are possible, but buffer is not forced to be zero terminated if on the stack. I'm pretty sure this was a result of copy-and-paste code where buffer used to be a static with a guaranteed zero, but it made me find several other places where similar things were happening.
I had started using a Q_strncpyz() function a while ago that guarantees a trailing zero and doesn't require the -1, but it turned out that between code I had written a long time ago, and code that either Cash or Brian had added, there were still a lot of normal strncpy calls around. A lot of them were wrong, too. Either missing the -1, or missing the dedicated 0 fill in.
Crashtest #4 from Jim Paris was a variation on the first part of #3.
Only one of these attacks so far has been a server crasher, but I have been giving the bounty for anything that immediately kicks all the players. I probably won't give it for attacks that only overflow some lagged clients, but I'll evaluate as they happen.
I am off to E3 now for a bunch of PR silliness, so if crashtest goes down, it won't be back up for a while...
* fixed crashtest 4
* fixed crashtest 3
* fixed jumping-over-item pickup prediction error
* made "Couldn't load sound" a developer only warning
* fixed demo recording not including portal surface entities
* precache grenade bounce sounds
-----------------------------------------
John Carmack's .plan for May 19, 1999
-----------------------------------------
Now that all the E3 stuff is done with, I can get back to work...
I was stuck in a room the entire time doing press interviews, but it seemed to have gone well. It was mentioned to me that there were a few people on the show floor with forged badges that read "John Carmack -- Id Software". As if forged email / irc / icq isn't enough of a problem. Sigh.
The "download" crashtest was first reported by Rick Hammerstone. That was a pure dumbass mistake on my part.
I should be sending the accumulated crashtest bounties off tomorrow.
The plan right now is to have an update release next week that will have lots of bug fixes and cheat protections, but not too many new user visible features.
I finally got around to implementing dual processor acceleration today. I still have a couple issues to resolve and some more rearranging to do, but it is giving 20%+ speedup right now in a worst-case situation for it.
When completed, I expect the average speedup to be in the 40% to 80% range, depending on what is going on and the video configuration. Scenes with lots of dynamic lighting and lots of sounds and other client processing going will show the largest speedups. It helps the slow scenes more than the fast scenes, which is basically what you want.
I am going to shake this out with the Windows (NT) code first, but it should definately make its way to the linux port eventually.
I know SMP is a que for all the BeOS folks to ask about ports, so I'm going to head that off: Be has all the code for Q3 (and Q2, for that matter), and a version of Q3test should be available by the time they ship a release OS with OpenGL hardware acceleration.
There will probably also be an SGI-irix port.
Regarding both of those ports: they are not supported ports, and will be maintained by volenteers (like the current MacOS X port). Update releases will lag the official ones, and we won't committ to ANY dates.
I am doing all of my development on intergraph and sgi-NT hardware, but when I have everything rock solid, I will give Nvidia and ATI's NT drivers a try. I would be somewhat shocked if they didn't explode -- I doubt multiple threads playing occasional tag team on a context has been well tested.
True, only a tiny fraction of our players (probably less than 1%) will be able to take advantage of this, but I consider SMP usage to be an important technology to nurture over the coming years.
The top of the benchmark chart should be an SMP system (assuming the NT drivers have all the optimizations of the '98 drivers), and it will also be possible to build a reletively cheap SMP system (say, dual 400's) that outperforms the best single processor system.
-----------------------------------------
John Carmack's .plan for May 22, 1999
-----------------------------------------
The SMP support is solid enough to play with now. The only feature that is still broken is light flares.
As a happy consequence, some of the cleanup work I did for SMP gave a couple percent speedup even when running without the separate thread.
On my development system, a dual 300 mhz intergraph realizm II, the low res timedemo scores went from 27.8 to 37.8 with "r_smp 1". This is only a 35% average speedup, but at some times (lots of dynamic lights in complex scenes) the speedup is 90%+. Gameplay is noticably smoother.
The rendering thread is almost always the blocking factor, so the faster the card and OpenGL driver, the larger the speedup will be.
This is explicitly a two thread producer / consumer, so there is no benefit to more than two processors. The app is well behaved, using sleeping syncronization so that you usually still have half a processor free for other operating system functions.
Hopefully we will be able to test with some fast consumer cards sometime soon.
------
A lot of people asked what was done differently this time vs the last time I tried (without benefit) to use SMP.
My original attempt was to make a DLL that intercepted all OpenGL calls and let a separate processor execute them. The benefit would have been that all OpenGL applications could have gone faster. The problem was that the bandwidth required to encode all the commands was enough that the processor overhead was as much as it would have taken to just do the geometry on the main processor.
It would have still been a win if the geometry side was doing lots of work, like multiple lights, user clip planes, and texgens, but for the vast majority of geometry, it didn't balance out. If someone wanted to try that using the PIII or AltiVec streaming memory operations, it could probably still work.
The current SMP code is implemented directly into the renderer, and a lot of things were moved around and double buffered to allow it to use data in place, instead of having to copy it off.
------
Some people expressed surprise that Quake3 wasn't threaded already.
Threading has been presented so often as the "high tech" "cool" way to program, that many people aren't aware of the downsides.
A multi-threaded program will always have somewhat lower throughput when running on a single CPU than a single threaded program that polls in explicit places. The cost of a context switch at the cpu level is negligible, but the damage that it can do to the cache hierarchy can add up to a noticeable amount in bad cases.
The much larger problem is that you lose tight control over when things occur. If the framerates are low enough, it isn't a huge issue, but for applications running at 30+ fps, you really don't want to trust the OS scheduler to coordinate multiple threads and have them all get in every frame. Yes, with explicit sleep() calls you can sort of get it working, but at that point, you might as well not be using threads.
A good example of not-quite-in-sync issues in the windows mouse performance. A PS/2 mouse only samples 40 times a second, so when you get an app updating at around that speed, you will get 0/1/2 scheduling variances.
They are also not terribly portable, and a pain in the ass to debug.
-----------------------------------------
John Carmack's .plan for May 26, 1999
-----------------------------------------
* basic joystick controls, some work still needed for advanced controlers
* r_dlightBacksides 0 option
* forced cvar_restart when version changes
* fixed some flare-in-fog problems
* fixed skin color in menus
* print obituary message even when you are the killer, so all kills get an entry in the logfile
* fixed bugs in line token parsing when quotes or commands aren't white space separated
* multiprocessor acceleration "r_smp 1"
* increase menu dimming
* increased rocket damage radius from 120 to 150 units
* check for running server in all server commands (dumpuser, etc)
* new cvar cheat setup -- by default, only archived variables can be changed when not cheating
* "cg_drawstatus 0" only removes status bar
* "cg_draw2d 0" removes all 2d
-----------------------------------------
John Carmack's .plan for May 27, 1999
-----------------------------------------
* enable scissor test properly
* archive r_lodBias
* cg_draw3dIcons 0 option
* data cheating protection
* userinfo renamed to clientinfo, added state and current server address
* don't forward commands to a server when playing demos
* fixed NULL extension on dir command
* added one more shotgun pellet
* added CG_Shutdown for cgame cleanup
* fixed jitter in rising smoke
* increase minimum time before reusing an entity slot
* soundinfo reports current background streaming file
* changed IPX separator to . from :, moved port processing to system independant code
* auto port scan wasn't updating the net_port cvar
* attack button presses reset inactivity timer now
* increased the forced respawn time from 10 to 20 seconds
* show smp on gfxinfo, slight reformat
-----------------------------------------
John Carmack's .plan for May 30, 1999
-----------------------------------------
For the past couple of weeks, I have been spending some development time on linux, and for the first time on a non-NEXTSTEP unix platform, I have actually been enjoying it.
While Id has been supporting linux since the Doom days, I have not personally been much of a linux user -- it was always ddt or zoid doing the actual coding and testing. Every year or so I would install a linux distribution and play around with it for a few days, but I would always leave feeling that it was still pretty crude (UI wise) compared to the NEXTSTEP UI I was used to, or even what I had used on other commercial unix workstations and windows.
There have always been a ton of reasons to like linux, but the user interface was enough of an issue that I couldn't buy into it completely.
The gnome user environment in Red Hat 6.0 is finally at a level that I consider it a valid alternative to commercial desktop environments. Overall, its still not as smooth, consistant, or complete as windows or the mac, but is does have its strong points, and things seem to be progressing quite rapidly.
Its still not something you would give to a purely casual computer user, but I won't be surprised if even that changes in a couple years.
CodeWarrior for linux is also a significant aspect of my enjoyment. Its a sort of crappy 1.0 port with a lot of little issues, but the editor works well enough, which is the important thing for me. I have never been able to stand vi or emacs for long enough to become proficient in them.
The code that I have been playing with most is the matrox g200 GLX driver.
Matrox is the first of the major 3D chip vendors that has had the guts to publicly release register level documentation for their 3D chips.
An accelerated X windows OpenGL driver has been put together with this by building on top of the existing Mesa and GLX projects.
It actually runs quake, quake2, and q3test. It doesn't run them FAST, but the quality is good, and I am impressed nonetheless. It is bordering on playable with all quality options set to the minimum on a fast computer, but it still has a ways to go before casual users should take a look at it.
It is steadily improving, and I hope Matrox will be pleased enough with the progress that they will release the documentation for their setup engine to go with the rasterizer.
In testing q3 on it, I noticed that with picmip set to 0, textures would get corrupted and it would never settle on a working set. The current Apple OpenGL drivers also have exactly this problem.
The cool part is that this driver is completely open source. I downloaded the project code, browsed through it a bit, and changed two lines of code to fix the bug. That RULES.
The next thing is sort of funny. I had been suspecting that a lot of the OpenGL drivers were clearing the entire depth buffer for each of the 3D icons on the status bar, instead of limiting it to the scissor region. I added code to the g200 driver to explicitly crop the clear region in the driver, but it never got executed. A little more investigation showed that I had been making an improper assumption for years -- scissor is not enabled by default. Doh.
Ever since noticing that glquake cleared the screen borders when the view is sized down, I had been operating under the assumption that intergraph just had a bug in their drivers. I had double checked that glClear was supposed to be limited to the scissor region, so I thought they were just messing it up.
Now I know that I was just being an idiot about that for the last three years... With scissor enabled, most of the cards got a few percent faster.
* dynamic curve level of detail. r_subdivisions determines the maximum level of detail, r_lodCurveError determines how quickly polygons are pulled out with distance
* devmap sets cheats 1, map sets cheats 0
* change weapon item upscale to 1.5 instead of 2
* always toss items forward, even if looking up or down
* draw ammo in grey while weapons are reloading
* change railgun shader while reloading
* fixed head models not showing proper skin
* skip all shell eject code when cg_brassTime 0
* fixed sound memory overallocation
* profiling and rearrangement
* fixed dead spectator bug
-----------------------------------------
John Carmack's .plan for Jun 03, 1999
-----------------------------------------
Whee! Lots of hate mail from strafe-jupers!
Some reasonable messages have convinced me that a single immediate jump after landing may be important to gameplay. I'll experiment with it.
Strafe jumping is an exploitable bug. Just because people have practiced hard to allow themselves to take advantage of it does not justify it's existance. When I tried fixing the code so that it just didn't work, I thought it changed the normal running movement in an unfortunate way.
In the absense of powerups or level features (wind tunnels, jump pads, etc), the game characters are supposed to be badasses with big guns. Arnold Schwartzenegger and Sigourney Weaver don't get down a hallway by hopping like a bunny rabbit.
This is personal preference, but when I play online, I enjoy it more when people are running around dodging, rather than hopping.
My personal preference just counts a lot. :-)
btw, here are the current weapon effects:
gauntlet: 50 pts, 400 msec / punch
machinegun: 10 pts, 100 msec / shot
shotgun: 11 pellets of 10 each, 1000 msec / shot
rocket launcher: 100 pts direct hit, or 100 pts splash damage falling off
over 120 world units, 800 msec / shot
plasma gun: 20 pts direct hit or 15 pts splash damage over 15 units,
100 msec / shot
railgun: 100 pts, 1500 msec / shot
lightning gun: 8 pts, 33 msec / trace, max range 768 units
grenade launcher: 100 pts direct hit, or 100 pts splash over 150 units,
800 msec / shot.
bfg: 40 pts instant splash damage over 100 units, 100 msec / shot
flamethrower: to be determined, but short range / wide angle
Splash damage is calculated from the edge of the player's box, unlike
quake1, where it was calculated froom the player's origin.
* ignore cl_maxpackets on LAN
* changed cl_packetdup to 1 by default, and archived
* defaulted com_maxfps to 100 and archived, automatically disabling during timedemo. It was possible to lag out some client connections on ultra fast systems even with cl_maxpackets set fairly low due to a huge number of individual commands being created
* 250 msec minimum time between landing and jumping again. I hate having players bouncing around all the time...
* fixed bug with large r_picmip values (white shotgun sight bug)
* new lightning and rail beam drawing
* player torso twitches with pain sounds
* r_drawstrips changed to r_primitives and archived, with changes: default "0" uses glDrawElements if compiled vertex arrays are present, or strips of glArrayElement if not. "1" forces strips, "2" forces drawElements, "-1" skips drawing
* increased rocket speed to 900 from 800. decreased direct hit damage from 120 to 100. splash damage same as 1.05
* removed sound-in-use dialog, auto skip after second try
* made userinfo persistant on server across level changes
* allow different servers to respond to a challenge, allowing redirecting server proxies
* notice bad ip addresses for connection: 192.1234.123.122
* removed neck length pivot to prevent view poking into low subdivided curves. Also make aiming when looking up or down more precise.
-----------------------------------------
John Carmack's .plan for Jun 27, 1999
-----------------------------------------
For the past couple years when talking to chip makers about new 3D features, 3D texture maps would always come up, and I would have to mutter something like: "I think it is a good thing, but I can't give you a really awesome example of using it. 3D Noise functions. Stuff like that."
I have the awesome example now: Lighting.
Through the entire development of Q3, I have been wanting to do dynamic lighting of the world differently. Many of the efficiencies gained by Q3's new map format result in larger and larger lightmaps while it reduces polygon count. That meant that even small dynamic lights may force a lot of work to generate new lighting textures.
I had been wanting to find a way to trade off additional rendering passes for less CPU time.
This is a perfect example of me outthinking myself.
I knew it would have to be something with projecting a light spot texture onto the world geometry, but I am keenly enough aware of the issues and limitations in projecting a 2D texture onto 3D geometry that my mind kept looking at worst case scenarios -- how do you project a continuous texture onto the inside of a dome with the proper intensities, and some situations with curves. I thought it would require some non-trivial per triangle analysis and operations, when I really just wanted something that could be done on a per-vertex level.
Recently, the 3D texture insight hit me.
Create a 3D texture of, say, 32*32*32 texels and fill it with your light attenuation function, bright point in the center, fading off to black at the edge. Set it to clamp mode, so any values beyond the edges stay black. Set up a texture coordinate generation matrix to position the light relative to your models. Its a simple translate and scale. Set up lighting calculation to only generate a value based on the relative angles, ignoring attenuation. Use the resulting modulated texture mapping as your "lightmap" to modulate another texture pass, or add to a previous lighting pass.
This has several important benefits:
It looks good (and identical) on both small and large polygons. Vertex based lighting needs an appalling amount of tessellation to avoid visibly triangulation artifacts. Even if you tessellate to the same sample density as your lightmap grid (hundreds of thousands per map), vertex lighting still looks worse, because it is a triangular instead of bilinear interpolation.
It can be done completely by upcoming lighting and texgen hardware, but is efficient in software implementations.
It can be shadowed by either stencil volume shadows or shadow buffer tests.
Post Q3A, I plan on doing a research engine that is fully dynamically lit and shadowed instead of using lightmaps.
Ok, but we don't have 3D textures in any hardware now, so this doesn't help me for Q3A.
I continued thinking along complex lines, like "Any planar slice of a spherical 3D texture will be a 2D circle, and the three triangle points can occupy any location on their respective rings".
It still looked like a big mess.
I got tired of thinking about it, and just started coding in some infrastructure to do extra lighting passes. I just made a gradient circle texture for the lighting, and generated the texture coordinates by just translating and scaling X and Y from the world coordinates.
I got it running, grabbed a rocket launcher, and fired down a hallway. It looked just fine. I was stunned. I had been thinking about complex ramifications of weird edge cases for the past year when all it took was a couple hours of programming and the simplest possible approach to make it work decent. Sigh. :-)
The only real addition required was a fading of the light contribution with Z distance from the surface. The downside to this hack is that while you get a nice ball of light moving over floors and ceilings, you only get a wash of light on walls. In hindsight, I can analyze this in the context of our games and say "Almost all movement and targeting takes place in the XY plane in a FPS, so accuracy along the Z axis is not necessary."
The somewhat more obvious change to dynamic lighting that wasn't related to the projection issue is that I have it currently set up as a postprocess pass, rather than an additive pass on the lightmap. Adding to the lightmap is more "right", but it really complicates the multitexture implementation, and on some surfaces, the lightmap is actually done after the base texture, so it couldn't be added to. The end result is that brighter areas are changed more by dlights than dark areas. I don't feel TOO bad about that, because its not like the dlight is much of a proper lighting simulation even in the best case...
* fixed tourney restart
* fixed jittering on plats
* fixed ref use without a world
* new default image that lets you see mapping coordinates
* fixed reliable sequences on restarts and demos
* allow maxclients to change between levels
* randomize shell ejection start position and angles
* display attackers head after every wound
* added gamma and overbright support to mac version
* removed table from sound mixing
* remove smoke and blood puffs when you run through them
* set cheats to 1 on disconnect
* shader sort value is now floating point
* new trajectory type "TR_INTERPOLATE", skip interpoaltion for other types
* fixed door open timing
* fixed bug with >8 portal areas
* added area print to r_shwocluster
* removed all the partial shader match cruft, fixing a crash-on-load
* fixed bug with personal shadows not being setup properly
* nomipmap shader parm split and expanded:
nopicmip : ignores r_picmip so image will always be high res
nomipmap : forces a single level texture, used for console font
* shrank sound mixing buffer for better caching
* new shader option: q3map_lightsubdivide. larger values make q3map -light proceed faster
* new vertex array interleaving
* fixed bmodels not counting patches in bounds
* fixed patch sphere culling on rotating entities
* cg_simpleItems now draws sprite items
* fixed serverid being 0 when map is started from cmdline
* fixed UI on mode changes
* replaced 1280*960 mode with 1280*1024
* test all rotated orders for tristrips from faces
* fixed RB_SurfacePolychain to not duplicate vertexes
* changed planar face surface type from a convex polygon to a general collection of coplanar triangles
* change renderer to use bmodel surface list instead of tree
* changed areabits pointer to areamask array
* fixed bad loop when client reliable message overflows
* fixed swapinterval after vid_restart
* wall mark clipping moved out of cgame, extended for better wrapping over multiple brushes
* picmip defaults to 1 under all cases now
* automatic curve LOD grouping
* duplicated SCR_ functions into cgame
* implemented Micahel Julier's optimization work
* implimented Michael Gold's SMP patch
* new reliable command transport
* use ENTITYNUM_NONE and ENTITYNUM_WORLD constants, which are now in MAX_GENTITIES range for safe net transport
* wait for attack released after respawning before firing
* added a delay before moving everyone to intermission spot
* fixed bug with regibbing of gibbed body ques
* blood trails behind gibs
* changed localents over to trajectories
* removed world as entity zero, clients are now 0 to MAXCLIENTS-1
* changed game interface functions to use clientnums instead of pointers
* removed speculative usercmd_t from user packets
* new item pickup code, fixed silent item pickup on grazing hits, and expanded the pickup range by 20% without changing the physical bounding box
* allow a single "quick jump" without delay
-----------------------------------------
John Carmack's .plan for Jul 03, 1999
-----------------------------------------
AMD K7 cpus are very fast.
Some timedemo numbers (a new demo, not comparable to previous scores):
Run at 640*480*16 bit color to emphasise the cpu/driver performance rather than the hardware fill rate.
[code]
K7-600 K7-550 PIII-500
TNT2 ultra 16 bit 73.9 68.5 53.8
Voodoo3 3000 16 bit 70.5 65.2 46.0
[/code]
This is with K7 optimized drivers vs seperate PIII optimized drivers.
There is still wiggle room there in that it is possible that more effort was expended to make the AMD drivers perform better. That is perfectly valid from a consumer's point of view, but muddies the technical CPU comparison.
On identical code run on the systems, there was some more interesting data:
On my map processing tools, the K7 was faster than the PIII, but only slightly more so than the reletive clock rate increase. I would guess that this is due to larger data sets that don't fit in cache as well.
On the matrox OpenGL drivers, which have not been optimized very much and (to my knowledge) contain no PIII specific code, the K7 was a LOT faster.
The bottom line is that I feel comfortable standing behind the statement that the K7 is faster than the PIII. I will have to wait for some stuff to come out of NDA to provide a more detailed technical analysis.
Architectural cleverness is all well and good, but if AMD can't keep the clock speed up with intel, they will still fall behind. A 700 mhz PIII would probably find a lot of applications (especially integer apps) where it would outperform a 600 mhz K7.
* stabilized cg_showfps
* added append support to module file opening
* automatic logging of game scores to games.log
* fixed guantlet firing action
* force a vid_restart on WM_DISPLAYCHANGE messages
* fixed sticking on stairs on very fast framerates
* fixed sticking on stair when jumping
* fixed sticking in corner while falling physics bug
* fixed slide down steep slope physics bug
* r_showimages texture use debugging tool
* cg_freezeDemo cvar
* cg_drawSnapshot cvar
* fixed warnings after demo playback
* changed "stopdemo" to "stoprecord"
* fixed phantom windows on task bar after exit
* check for unset player animation
* fixed the snap-down-look-up bug with very high sensitivities
* reduce inflicted damage by handicap
* all pmove results as events for proper demo playback
-----------------------------------------
John Carmack's .plan for Jul 24, 1999
-----------------------------------------
I was in San Jose for the past week. The original idea was to go into "hermit mode" holed up in a hotel room and get a lot of work done without any phone calls, email, or coworkers to distract me, but I wound up with meetings scheduled every day with various valley companies. Next time I want to try that, I'll pick some state like Montana... :-)
The top priority was getting the virtual machine done, but I hoped to also get some more of the map data optimizations completed. I definately underestimated how big of a hole daily meetings would punch in the amount of work I could accomplish.
On wednesday I was sweating a bit, not sure if I would have the VM finished in time, but it all came together in the last two days.
The virtual machine interpreter is now completely functional, and cgame can switch between being loaded as a binary .dll or an interpreted .qvm at the change of a cvar.
The basic setup is that I have a modified version of the lcc compiler that generates assembly files that are processed by a new tool "q3asm" into a .qvm file that can be interpreted by q3. You can still use normal dll's for debugging, then release interpreted code. You can release a binary dll if you need some native system services (some networking stuff, for example) or are doing very compute intensive work, but I strongly encourage everyone to try to use the virtual machine.
The interpreted code has two prime benefits: portability and security.
User mods done with qvms will automatically work on mac, linux, and any other oddball ports that get released.
A qvm program cannot modify anything outside its private address space, and the "system calls" to the game strictly limit what can be touched. I was scared about the binary dll's in Q2, but rationalized it to myself that people running public servers should be aware of the dangers. With Q3 allowing client side programming, it just needs to be safe for everyone.
Packaging is also improved somewhat, because the virtual machine programs can be included in pak files and are automatically handled through the search path.
Unfortunately, even after doing most of the straightforward optimizations, the interpreter is causing a 20% slowdown in timedemos right now.
I am pretty committed to running cgame interpreted, but if I don't get a significant speedup, we may have to leave the server side game module as a native dll. The ui module can obviously be interpreted.
There are lots of paths I can take to get the performance up:
Write more efficient cgame code. I will definately be looking at this. A lot of things that just didn't matter at all when in native code now add up enough that they should be fixed. I want to avoid flexibility tradeoffs if possible.
Move more calculation from the cgame to system calls. I have already done this for the obvious things like memset, matrixmultiply, etc. Any future work will involve restructuring cgame code to focus lots of work into simple stateless functions. The trick is to leave all the flexibility in the cgame while moving the work.
Implement LCC compiler optimizations. I doubt I will pursue this, unless there are existing optimization projects based on LCC with minimal backend changes required.
Implement more complex interpreted instructions and peephole combine the bytecodes into the new instructions. I think this has some promise.
Assembly code the interpreter loop. The compiler is generating good code, but there is still room for gain. I don't want to pursue this until the high level optimizations are wrung out.
Load time compilation from bytecode to native code. This would bloat a lot, especially on RISC cpus. I'm not sure that a straightforward implementation would have significant speed benefits over an assembly interpreter loop, and I won't have time for writing an aggressive native optimizer.
Deal with the performance cost and optimize the renderer some more to compensate.
-----------------------------------------
John Carmack's .plan for Jul 29, 1999
-----------------------------------------
* log all client transitions, item pickups, and kills
* changed joystick axis to act just like arrow keys so they can be bound in the controls menu for strafing. Yes, this does remove slow walking from joystick movement, but it makes everything a lot cleaner.
* fix up PantherXL trackball support
* removed bad clear command from dedicated console clear button
* tournement queuing of spectators to enter the game
* track wins and losses as long as you stay on a tourney server
* spectators are now in fly mode instead of noclip, and use teleporters
* pass serverTime instead of msec for command timing, prevents timescale cheating
* track dual eventParms on player state
* draw crosshair and name in spectator mode
* fixed rcon
* r_colorMipLevels texture visualization tool
* don't allow weapon select and cycle when in follow mode
* archive cl_yawspeed and cl_pitchspeed
* don't draw place line on scoreboard when spectating
* fixed console chatting during intermission
* better recursive error handling
* fixed curve surface flags (no impact when landing on a curve bug)
-----------------------------------------
John Carmack's .plan for Jul 30, 1999
-----------------------------------------
I apologize for not getting the Mac version released this last time.
All conspiracy theories aside, here is what actually happened:
I had my San Jose travel scheduled a while ago, and we were expecting to have the release done before I left. We didn't quite make it, and nobody else at the office knew how to build the mac version after Cash made some last minute changes.
When I got back, Graeme was taking off to move his family down here. Graeme is in charge of building all the installs for our releases.
I considered just tossing new executables with the latest fixes for everything, but some data has changed, and it just isn't worth the hassle right now.
We will be making proper new releases for all architectures monday night when he gets back.
* fixed marks fading properly in fog volumes
* show weapon in fov >90, adjusting position down as needed
* allow run/bob variables to be changed in non-cheat games
* update scoreboard info while at intermission
* fixed angles on resetplayerentities, corrects twitch on players as you come through a teleporter or respawn
* print "waiting to play" for tourney spectators
* fixed tied rank with 0 score and spectators
* return to roaming spectator when a followed client quits or spectates
* release windows cursor when running windowed and the console is down
-----------------------------------------
John Carmack's .plan for Aug 01, 1999
-----------------------------------------
* changed RF_PORTALSURFACE to an entity type
* add sprite indexes in strip order
* changed builtin models to refentity types
* prevent respawn until resting on ground
* fixed unecessary dlight shader checking
* no splashe when lg hitting sky
* removed cvar_restart on version change, caused problems with dedicated server startup
* fixed stupid error that made skies double render
* fixed stupid error that made skies draw an uneeded bakground pass
* support add-mode multitexture for skies
-----------------------------------------
John Carmack's .plan for Aug 16, 1999
-----------------------------------------
As I mentioned at quakecon, I decided to go ahead and try a dynamic code generator to speed up the game interpreters. I was uneasy about it, but the current performance was far enough off of my targets that I didn't see any other way.
After getting over being sick the start of the week (someone from QC must have brought me a flu present), I decided to dive in to it.
At first, I was surprised at how quickly it was going. The first day, I worked out my system calling conventions and execution environment and implemented enough opcode translations to get "hello world" executing.
The second day I just plowed through opcode translations, tediously generating a lot of code like this:
[code]
case OP_NEGI:
EmitString( "F7 1F" ); // neg dword ptr [edi]
break;
case OP_ADD:
EmitString( "8B 07" ); // mov eax, dword ptr [edi]
EmitString( "01 47 FC" ); // add dword ptr [edi-4],eax
EmitString( "83 EF 04" ); // sub edi,4
break;
case OP_SUB:
EmitString( "8B 07" ); // mov eax, dword ptr [edi]
EmitString( "29 47 FC" ); // sub dword ptr [edi-4],eax
EmitString( "83 EF 04" ); // sub edi,4
break;
case OP_DIVI:
EmitString( "8B 47 FC" ); // mov eax,dword ptr [edi-4]
EmitString( "99" ); // cdq
EmitString( "F7 3F" ); // idiv dword ptr [edi]
EmitString( "89 47 FC" ); // mov dword ptr [edi-4],eax
EmitString( "83 EF 04" ); // sub edi,4
break;
[/code]
(yes, I could save a few bytes in those opcodes by moving the sub edi, but I am trying to leave the subs at the bottom and the adds at the top, in case I want to add a peephole optimizer)
I was writing test programs as I went, so I thought it was still going quite well. I quit for the day with only six opcodes left to write.
Today I got in, wrote the last opcodes, and started running the full cgame module.
The first problem was obvious: the loading screen graphics came up with the default image instead of the text font. I quickly found and fixed a problem with system call return values.
It was then able to get into the game, but the FOV was clamped out to 160, and it crashed when you fired your gun. That turned out to be my failure to fix the operand stack correctly in the rarely used structure copy opcode.
The next problem was a little more distressing. You could run around the level, but all the items had an X coordinate of 0. This took a little while to find, and turned out to be a twitchy case of sometimes getting an extra value on the operand stack.
At that point everything looked like it was working perfectly. I was ecstatic. I started running timedemos to see what the performance looked like. Then I ran a demo without timedemo on, and near the end of the demo it crashed. Then I found out that if you play in a level for a few minutes, it either gets an error of some kind, or crashes.
I was quite unhappy about that. Debugging a non-deterministic crash in generated code. Joy.
What I wound up eventually doing was to make gigantic log files of all system call interactions and compare the compiled code against identical runs with the interpreter. I had to tweak a few things to make the processing exactly the same between them, but it let me find where things first started to diverge. It turns out I was letting the top of my compiled code's local stack creep down a bit with each call. Let it run long enough, and it started hitting important things. If I had logged stack pointers instead of parameter values, I would have found it a whole lot quicker...
Now, I am pretty confident that it is correct.
The generated code is pretty grim if you look at it, in part due to the security measures (mask and add for each load/store), and in part due to the fact that it is a straight bytecode translation:
[code]
06214DD0 83 C7 04 add edi,4
06214DD3 C7 07 00 4D 0A 00 mov dword ptr [edi],0A4D00h
06214DD9 8B 1F mov ebx,dword ptr [edi]
06214DDB 81 E3 FF FF 1F 00 and ebx,1FFFFFh
06214DE1 8B 83 30 00 DC 05 mov eax,dword ptr [ebx+5DC0030h]
06214DE7 89 07 mov dword ptr [edi],eax
06214DE9 83 C7 04 add edi,4
06214DEC C7 07 2C 00 00 00 mov dword ptr [edi],2Ch
06214DF2 8B 07 mov eax,dword ptr [edi]
06214DF4 01 47 FC add dword ptr [edi-4],eax
06214DF7 83 EF 04 sub edi,4
06214DFA 8B 1F mov ebx,dword ptr [edi]
06214DFC 81 E3 FF FF 1F 00 and ebx,1FFFFFh
06214E02 8B 83 30 00 DC 05 mov eax,dword ptr [ebx+5DC0030h]
06214E08 89 07 mov dword ptr [edi],eax
06214E0A 8B 07 mov eax,dword ptr [edi]
06214E0C 29 47 FC sub dword ptr [edi-4],eax
06214E0F 83 EF 04 sub edi,4
06214E12 83 C7 04 add edi,4
06214E15 C7 07 40 00 00 00 mov dword ptr [edi],40h
[/code]
Code bulk is also up there, at about 5x the bytecode version. There is definately some savings to be had with better opcode selection, but no more than 30% or so at best.
Performance is within my tolerance now:
Q3demo1 dll: 52.9
Compiled: 50.2
Interpreted: 43.9
Q3demo2 dll: 50.1
Compiled: 46.5
Interpreted: 38.7
I will probably work a bit more on performance, but that is the ballpark that it will be in. 5% speed hit in most levels, somewhat more in the big open arenas. Next week I will be getting the other modules set up for running in the virtual machine and see how their performance is.
It is a pretty cool setup - you can have some modules as dlls, some as interpreted bytecodes, and some as compiled bytecodes. We will leave the user interface interpreted to save memory.
Tomorrow I am going to get all the byte order issues worked out for powerPC. The interpreter doesn't even work there yet because of inline constant byte order issues. Fixing that will slow the interpreter a little more, but that shouldn't be any problem, with the performance oriented stuff being compiled.
Doing the PPC compiler will be a bit messier because the tools aren't as nice, and the fact that it will involve a whole lot of Mac crash/reboot cycles before it stabilizes, but I think I know what to watch out for now.
VC6 crashed on me about a dozen times in the last few days, probably due to my having show-code-bytes on in the dissassembly window, but it was still pretty damn useful through the whole process.
I am curious to see how the RISC code bulk turns out. The instructions are going to be longer, but all the constants can be held in registers. Should be interesting.
I don't think I am going to be in any hurry to do MIPS/ALPHA/SPARC code generators. One or two code generators and execution environments is educational, but that will be more than enough for me. If we do port to other architectures, they can still run with the interpreter or binary modules until someone else gets up the inclination to do a code generator.
-----------------------------------------
John Carmack's .plan for Aug 26, 1999
-----------------------------------------
The current plan is that we will have another test release around the middle of next month. This version will be running game/cgame/ui with the virtual machine, and will include single player play against bots. No new maps.
I will be releasing the source for all the VM modules with it, and setting the executable up so that it will allow modified modules to be used.
The modified LCC and q3asm will be available both in source form and precompiled, so a professional development environment will not be requried. Using MSDEV to debug binary dll's does make exploration a lot easier than adding prints to interpreted code...
Some minor porting work on the tools will be necessary to do development under linux. The effort would be greater for mac development, because the tools are inherently command line based.
The map editor and tools will not be released until after the game hits store shelves. To be completely clear: you are not legally licensed to create or use addon maps with the test.
I am hoping that this public review will turn up bugs before we complete the game. 50k lines of code is quite a bit to go over, but people familiar with previous games will have a good head start in the game module.
The best possible situation would be if exploration of the code evolves into a tier system, with either moderated or limited access lists that I can follow without being swamped. I can't afford to be too involved in helping everyone figure out the basics. I have plenty of confidence in the mod communities ability to work that out. :-)
---------
Some people have been mistaking memory swapping in 1.08 for network problems.
We did controlled, back to back tests against the previous versions, and the networking is identical if you have enough memory.
The addition of the new character model and all the new menu code and graphics has caused the game to begin to have some swapping problems on 64mb machines if you have all the quality options up high or are running other things.
I am looking into what I can do to reduce memory consumption for the next release, but in the meantime, you can turn down sound quality, geometry, or texture detail to get rid of it.
If you have less than 64mb, go buy more memory! The final game will have an option to run in less memory, but the graphics and sound quality will be a lot lower.
----------
I made a simple change in the file management that I think is clearly a big win.
Instead of scanning for pak0.pk3, pak1.pk3, ... pak9.pk3 in game directories, I now scan the entire directory for all .pk3 files, and add them to the search path in alphabetical order.
This gives us the same needed functionality we have now -- overriding things in pak0.pk3 with updated data in pak1.pk3, but it also gives a signficant benefit to the user community.
There has been a lot of requests to have textures inside maps like Quake1 did, but I was not willing to do that. Having the files separate saves an immense amount of duplication, and keeps the internal architecture uniform.
Now, you can just add all your new art into a pk3 (zip) file along with your map and users can just drop that single file into their quake3/baseq3 directory as a single operation.
This will also be nice for custom models, which require several component parts: legs, torso, head, animations, levels of detail, and skins.
So, if you are strictly ADDING media (models, textures, maps, etc), then you can just drop the pk3 files in the normal directory.
If you are REPLACING data (code modules, menus, etc), you should make a new game directory and put the pk3 there. Starting up with "quake3 +set game mygame" will make all the mygame pk3 files override anything in baseq3. You could do it by just naming your pk3 file "zzzstuff.pk3", but then you wouldn't have a way to run the game without the addons.
A prudent person might choose to put ALL addons into a separate directory and leave baseq3 pristine for official additions.
----------
Other stuff that has been done lately:
* don't clamp dedicated server or client times until much later -- prevents time resets under ordinary conditions
* fixed CG_ProcessSnapshots: cg.snap->serverTime
* only drift time on receipt of packets with a steady ping -- improve catchup after drops
* fixed players getting stuck together
* new pak file support
* derez mac resource file
* intro/loop option for music files
* fixed lerp-through-world when changing teams
* show ping and netgraph for spectators
* fixed timescale off by one problem
* tracked down player count wrong on server list
* info_spectator_start entity
* shader language change: clampTexCoord removed, added clampmap
* r_debugSort cvar for working on transparency sorting problems
* changed minimum cl_maxpackets to 15
* fixed can't-respawn-when-someone-is-on-your-body
* fixed dlighting over alpha tested surfaces
* z_stats lists all blocks >= given size
* fixed wasted model slots
* increased com_hunkmegs
* com_buildScript cvar to force loading all media and quit on errors
* fixed bad playerstate interpolation across teleporters
* converted local sounds to sfxHandle_t
* new fog code doesn't require subdivisions
* fixed sun positioning problem
* added fogging of triangle objects
* fixed devmap issue
* make g_log a filename instead of a 0/1
* g_logsync option to force a flush after each write must be set at time of log file creation
-----------------------------------------
John Carmack's .plan for Aug 28, 1999
-----------------------------------------
* spinning machinegun barrel
* changed q3data -origin option to -offset, defaulted to 0 0 24 for all player grabs
* removed second parm from -lod in q3data
* fixed 0 ping on last player killed before fraglimit
* better ping calculation right after transitions
* add time back to scoreboard
* sv_maxRate option to force all clients to play with a max rate. This can be used to limit the advantage of LPB, or to cap bandwidth utilization for a server. Note that rate is ignored for clients that are on the same LAN.
* fixed bad name vs name in tourney after first player left
* added hitch warning messages to server console
* new time clamping rules for net play
* avoid sending usercmds during connection
* send explicit heartbeats to the master server when a server transitions to or from empty or full
* shaders that aren't found will return index 0, but still keep the allocated slot to prevent rescanning if registered again
* use nextSnap for player prediction when available
* removed teleport dest invisible objects
* reduced client to server bandwidth by 35%
* changed logging for chats to guarantee parsing properly with names that conflict with commands:
from: G_LogPrintf( "%s say: %s"
to: G_LogPrintf( "say: %s: %s"
-----------------------------------------
John Carmack's .plan for Sep 01, 1999
-----------------------------------------
I have been working on our memory footprint for the past couple days. There are two types of paging that occur:
One time only paging that eventually settles down after you have run around the entire level a few times. This can be a result of having lots of data around that isn't actively used during the game, or was only used at startup.
True capacity misses, where the game is actually touching more memory during play than you have available. This is usually due to there just being too many textures and models.
Loading a large map with full resolution, 32 bit textures and mipmaps consumes almost 40 megs of texture space. The default of picmip 1 and 16 bit textures reduces that to six or seven (not all images observe picmip) in theory, but some OpenGL implementations keep a 32 bit version of the texture around even if the card only uses 16 bit (I consider this inapropriate), so it may still be twelve or more megs just for the images.
I have been able to save a couple megs off of the true capacity requirements by not rolling through some larger buffers when not needed (not smp and not dedicated server), and several megs more of one-time data mostly by moving a lot of static tables to dynamic allocation.
I can probably save another meg of true capacity and I think three or four megs of initialization data. The big thing that I might be forced to do is go to a skeletal animation system. I would hate to do that at this late a point in the project, but it would save about two megs per player model in the game.
In the process of chasing down the static memory hogs, I finally got around to starting something I have needed to do for years: learn perl.
I was scanning through a linker map file looking for large gaps in addresses, thinking to myself "this is one of those things you can probably do in three lines of perl code". I have many and varied excuses for why I have never gotten around to it before, mostly involving the fact that I have C parsing code that lets me get what I need done with only minimal headache when I do force myself to do some text file grovelling.
I decided my excuses weren't good anymore, and went out to the bookstore and grabbed the llama book. Many of you would have been amused seeing me go through the
print "Hello, $name!"
tutorial code as I did the examples in the first couple chapters. :-)
I got my task done, so now I just need to force myself to write little perl programs whenever a need comes up, until I get fluent with it.
* save 2.5 megs by reworking shader allocation
* save 1 meg by not double buffering backend if not smp
* convert all tr. arrays into pointers
* don't allocate as many snapshotentities when non dedicated
* new shader option: deformvertex move
* stackable deformvertex
* reduced lightning damage by 10%
* light emit from two sided surfaces
* reduced starting machinegun ammo in teamplay to 50 from 100
-----------------------------------------
John Carmack's .plan for Sep 02, 1999
-----------------------------------------
I have been getting a lot of requests for commentary on two subjects lately:
Nvidia's new geometry accelerated card with the funny name.
It is fast. Very, very fast. It has the highest fill rate of any card we have ever tested, has improved image quality over TNT2, and it gives timedemo scores 40% faster than the next closest score with extremely raw beta drivers.
The throughput will definately improve even more as their drivers mature.
For max framerates in OpenGL games, this card is going to be very hard to beat.
Q3's target of about 10,000 triangles a frame doesn't stress this card at all. If you want more polygons out of Q3, you can do:
r_lodBias -2 // don't use lower detail models
r_subdivisions 1 // lots more triangles in curves
r_lodCurveError 10000 // don't drop curve rows for a long time
I haven't looked at the stencil shadow stuff in a long time, but it gives the largest increase in triangle use (and a lot of fill rate as well):
cg_shadows 2 // turn on stencil shadows (if you have a stencil buffer)
Apple's new G4 systems.
The initial systems are just G4 processors in basically the same systems as the current G3. There will be some speedup in the normal C code from the faster floating point unit, and the Apple OpenGL has AltiVec optimizations, so framerates will improve somewhat. The limiting factor is going to be the fill rate on the rage128 and the bandwidth of the 66mhz pci bus and processor to main memory writes.
The later G4 systems with the new memory controller and AGP will have better performance, but probably still limited by the new 3D card.
After Apple gets all their driver tuning done, it will be interesting to try running timedemos at low resolution to factor the fill rate out. Apple has a shot at having the best non-geometry accelerated throughput, but it will still be tough to overcome a K7 with an extra hundred or so mhz.
On a purely technical note, AltiVec is more flexible for computation than intel or AMD's extensions (trinary ops), but intel style write combining is better for filling command buffers than the G4's memory streaming operations.
-----------------------------------------
John Carmack's .plan for Sep 06, 1999
-----------------------------------------
It looks like we are going to go to a skeletal model system. Jim Dose of Ritual had already started on an exporter from character studio, so we decided to just meet in the middle.
I implemented the loading and rendering support this weekend and tested it with a couple hand-inserted bones, so now we just need to write the glue between character studio and the new .md4 format.
The new format is bone based, but it is NOT hierarchial. Each vertex just has an arbitrary weighted list of the bones that influence it. Bones are just 4x3 matricies of floats.
A hierarchial skeleton has some advantages (angles instead of matricies, ability to do IK, etc), but this is a direct and simple replacement for our existing infrastructure that doesn't require any cached state per model instance.
A single .md4 file holds multiple level of detail surface sets, which all share the same bone frames.
In use, it is exactly like the existing models (interpolate between two frame numbers), it just saves a huge amount of space.
I used perl to generate my test data, and it was definately faster than having a separate msdev open and doing it in C.
I am trying to use parenthesis on all perl functions, but when I type "print", my fingers seem to have a flashback to applesoft basic fifteen years ago, and I wind up with bare quotes on prints and parens on everything else...
Does anyone know if there is an existing msdev syntax coloring file for perl? (no, I don't want to switch to a different editor!)
* md4 model loading and displaying
* removed clip models from cgame, use renderer models instead
* fixed mover pushing again
* fixed bug with culling of mirrors made of multiple faces
* fixed quad on spinning machinegun
* surfaceparm alphashadow - This causes q3map -light to check individual texture pixels on transparant surfaces for light shadowing instead of making the entire surface either cast or not cast shadows
-----------------------------------------
John Carmack's .plan for Sep 28, 1999
-----------------------------------------
Ok, obviously we didn't get a release out in the middle of the month...
We are still hashing out the single player game, so it still isn't immediately immenent.
* weapon switch animations at 20hz instead of 15hz, cuts switch time from 600 msec to 450 msec
* initial spawn flag for single player
* finished new fog code
* fixed walking underwater friction problems
* autosprite2 now selects the longest axis to pivot on, and allows any texture mapping, not just unity
* fixed autosprite on entities
* fixed lurching during low timescale
* reduced machinegun damage
* set clamp mode on 2D pics
* take hostname off of single player connect
* remvoed dlighting on skies and solid lava
* fixed lower body twitch when copytobodyque with a motion
* always show your score in second box if not in first
* sarge as default model
* com_blood 0 option for no gibs and no blood on hits
* mouse click aborts cinematic
* show tourney scores in all games, add fraglimit
* removed tripple bunny-hop protection, it was too arbitrary and didn't accomplish it's goal
* pump event loop during level loading
* added pass count to shaderlist
* default to CGEN_IDENTITY / CGEN_IDENTITY_LIGHTING based on blendSrc
* optional simplified blendfuncs: blendfunc
* new shader command: deformVertexes normal
* new shader command: tcgen vector ( ) ( )
* fixed fog on alpha tested surfaces
* reduced com_maxfps to 85
* defined shaders for menu and console backgrounds
* reset players on clientinfo transitions
* windows icons
* fixed powerups on spinning barrels
* removed some latency from lightning endpoint
* fixed lightning bolt drawing too far
* removed color clamping from entity lighting
* moved all 2D drawing to shader pipeline
* r_printShaders tool
* moved dlighting into world node descent
* pause when menu is up in single player
* fixed double EV_FIRE_WEAPON
* r_singleShader optimization tool
* some renderer optimizations
* better multitexture collapsing
-----------------------------------------
John Carmack's .plan for Sep 29, 1999
-----------------------------------------
I wrote this in answer to a question on the mac opengl programming list, but it is of general enough interest to programmers that I am repeating it here.
>You also mentioned display lists... Can you explain what some of the
>major things that should be rendered using a display list are? I see
>the importance of using them for characters (animation) and objects, but
>what about using them for the rest of the world, particles, and other
>things that are nifty.
This is not yet a big issue, although even pure software OpenGL's could perform some optimizations with display lists that aren't possible with vertex arrays. With hardware geometry acceleration, it can be an honest 4x improvement in throughput.
The important point is that once geometry acceleration becomes a primary target, practically everything will have to be rendered with display lists, or you will run into a nasty case of Amdahl's law.
In a busy Q3 battle, the triangle count may be split roughly evenly between character models and world geometry. Going from an empty scene to a pitched battle can result in a 50% performance drop if you are triangle limited. Not great, but livable.
If we kept the same ratios and designed for geometry acceleration with all the static world geometry in display lists, then the empty scene could have 4x the geometry and still be running the same speed. However, current OpenGL display lists can't really accelerate high quality skinned characters, so when an equal number of character polygons was in scene and passed through normal direct rendering, the performance would drop to 20% of the original. Unacceptable.
So, either you would have to use significantly different polygon counts in characters and the world, or some new API features would need to be defined. Nvidia has a skinning extension that gives some benefit, but still requires a character to be broken up into one static list per bone pair, instead of a single list for the entire character.
Rendering a few thousand particles or other procedurally generated triangles directly isn't going to be a big issue, but the bulk of the work is going to move towards static vertex data.
My advice for display lists is to use them for just raw vertex/color/texcoord data, and keep your state changes done with direct commands. This allows you to still sort display lists to minimze state changes, and prevents drivers from ever having to check state internally. Some hardware architectures can nicely encapsulate all state changes in a single dma buffer, but register sharing among different fields sometimes requires the driver to do manual masks, negating much of the async display list benefits. Texture swapping also complicates state changes inside display lists.
You want to make the display lists as big as practical, but there is a tradeoff betwen culling tightness and display list size.
-----------------------------------------
John Carmack's .plan for Oct 01, 1999
-----------------------------------------
* 10 crosshairs to select from (cg_drawCrosshair 1 - 10)
* cg_crosshairx / cg_crosshairy adjustment. I'm not convinced these are a good thing, because the crosshair is accurate in Q3 at the default position (unlike Q2, which had an offset firing position)
* more packet encryption
* join as spectator in all team games
* cg_predictItems 0 option to not do local prediction of item pickup
* rank players counting ties, so the third player when the lead is tied is third place, not second
* properly stack all status elements in upper right and lower right corners so they can all be visible
* faced q3map problem giving black corners in vertex light
* fixed +button4 not causing footsteps
* fixed bad groundplane clipping on angled jumppads
* show "snc" on lagometer when g_syncronousClients
* new shader command: rgbgen const ( )
* new shader command: agen const
* snap dropped item positions
* randomized offsets of bubbles
* fixed cgame restarts processing snapshots from 0
* proper setup for external/predictable player events
* fixed q3map vertex lighting bug after alphashadows
* do personal pain sounds on health transitions so they can never be missed
* timeout clear player events
* fixed hang in UI_ProportionalStringWidth
* quad event implicit on weapon fire
* fixed gamestate not retransmitting bug
* fixed timeout issue when paused
* subdivide command times if <15fps, fixing low com_maxfps physics exploits
* fixed footsteps playing when walking backwards
* new options at start of animation.cfg file: footsteps, headoffset
-----------------------------------------
John Carmack's .plan for Oct 04, 1999
-----------------------------------------
* allow cg_thirdPerson and cg_thirdPersonRange in games
* added execed .cfg files to journal file
* single pass plasma explosion effect to save overdraw
* rescaled and sized gib and mark blood to save a lot of overdraw
* fixed cg.time < snapshot time on vid_restart
* cl_showSend network debugging tool
* back to requiring a before commands on the console to distinguish them from chat messages. Tab completion automatically adds the slash.
* new tab command completion with complete to longest common, multiple match listing, complete first token, etc
* separate version check for game/cgame in addition to system
* show r_finish in gfxinfo
* never show self as attacker icon
* callvote vote - Caller automatically votes yes vote has a 30 second timeout each client can only call 3 votes a level vote is displayed on screen with totals
* renamed cg_gun to cg_drawGun
* box cull triangle soup models
-----------------------------------------
John Carmack's .plan for Oct 05, 1999
-----------------------------------------
* fixed steady snapshot test
* fixed incorrect 0 ping if past client messages
* fixed loser-disconnecting-at-tourney-intermission sorting problem
* general purpose flood protection, limiting all user commands to one a second by stalling the client, so the commands don't actually get dropped, but are delayed as needed
* replace headnode overflow with lastCluster
* fixed bad extrapolation on unpausing
* fixed player twitch on unpausing
* print client names on loading screen
-----------------------------------------
John Carmack's .plan for Oct 07, 1999
-----------------------------------------
* r_primitives 3 path for non-vertex array testing
* specify sex in model animation.cfg file
* proper dropping of failed bot inits
* removed identical pain sounds
* serverTime strictly increasing across levels
* added GL_DECAL multitexture collapse
* windowed mouse on mac
* fixed byte order issue with curve clipping on mac
* made com_defaultextension buffer safe
* fixed levelshot and added antialiasing to image
* don't clear bot names before kick message
* made servercommand sequences strictly increasing across level changes
* unpause on vid_restart
-----------------------------------------
John Carmack's .plan for Oct 11, 1999
-----------------------------------------
* handle window close events properly
* enable r_displayRefresh selection and feedback on mac
* avoid AGEN_IDENTITY when CGEN_DIFFUSE_LIGHTING
* colorized railgun muzzle flash, impact flash, and mark
* exactly synced animating textures with waveforms and collapsed all explosion sequences into single shaders
* removed unneeded black pass on hell skies
* fixed grenades sticking to steep slopes
* scan for next highest fullscreen resolution when exact mode fails (fixes SGI flat panel failing to init)
* all cgame cvars now have a cg_ prefix (crosshair, fov, etc)
* clear clientinfo before parsing configstring
* make all feedback voiceovers share the same channel
* fixed nodraw curves
* fixed obits from shooter entities
* fixed chat char issue
* separate gentity_t fields into sharedEntity_t
* reintegrated q_mathsys with q_math
* cg_forcemodel also forces player sounds
* unknown cmd commands don't chat
* fixed strip order on text quads
-----------------------------------------
John Carmack's .plan for Oct 14, 1999
-----------------------------------------
* make sure video is shutfown for errors on startup
* automatic fallback to vm if dll load fails
* compressed jump tables for qvm
* removed common qfiles.h and surfaceflags.h from utils and game
* don't load qvm symbols if not running with developer
* "quake3 safe" will run the game without loading q3config.cfg
* ignore network packets when in single player mode
* dedicated server memory optimizations. Tips:
com_hunkMegs 4
sv_maxclients 3
bot_enable 0
* fixed logfile on mac
* new time drifting code
* fixed file handle leak with compressed pk3 files
* q3data changed to remove shader references from player models
* throw a fatal error if drop errors are streaming in
* fixed com_hunkMegs as command line parm
* spawn spectators at intermission point (info_spectator_start has been removed)
* new sound channel for local sounds
* fixed follow toggle on bots
* don't write to games.log in single player
* fixed improper case sensitivity in S_FindName
-----------------------------------------
John Carmack's .plan for Oct 17, 1999
-----------------------------------------
The next release will be the full "demo" release for quake 3. It will include bots and a new, simple level that is suitable for complete beginners to play, as well as the existing q3test maps.
The timing just didn't work out right for another test before we complete the game.
We plan on releasing the demo after code freeze, when the entire game is in final testing, which will give us a few days time to fix any last minute problems that show up before golden master.
No, I don't have an actual date when that will be.
-
I got an iBook in on friday. It is sort of neat (I need to go buy an AirPort, then it will definately be neat), but it is currently way too slow to play Q3 on.
Apple's high end G3 and G4 systems with rage128/rage128pro cards and latest drivers are just about as fast as equivelant wintel systems, but the rest of the product line is still suffering a noticable speed disadvantage.
The new iMac has a rage128, but it is only an 8mb/64bit version. Still, with agp texturing it is a solid platform with performance that is certainly good enough to play the game well on.
Existing iMacs have 6mb ragePro video. ATI's windows drivers for the pro have come a long ways, and Q3 is plenty playbale on windows with a rage pro (ugly, but playable). On apple systems (iMacs and beige G3's), the performance is sometimes as low as HALF the windows platform. The lack of AGP contributes some to this, but it is mostly just a case of the drivers not being optimzed yet. The focus has been on the rage128 so far.
The iBook is also ragePro based, but it is an ultra-cheap 32 bit version. It does texture over AGP, but it is slooooow. I suspect it is still driver issues, because it is still very slow even at 320x240, so that leaves hope that it can be improved.
Another issue with the Apple systems is that Apple 16 bit color is actually 15 bit. I never used to think it made much difference, but I have been doing a lot of side by side comparying, and it does turn out to be a noticable loss of quality.
* new lg splash
* added channel number for local sounds so feedbacks don't override announcers
* removed scoreup feedback sound
* expand score tabs as needed for large scores
* fixed bfg obit
* fixed swimming off floors
* fixed swim animation when jumping in shallow water
* fixed first weapopn switch problem
* convert all joystick axis to button events (twist is now bindable)
-----------------------------------------
John Carmack's .plan for Oct 23, 1999
-----------------------------------------
An announcement:
We have hired Robert Duffy as a full time employee, starting in December.
He has been maintaining the level editor since the release of Q2, but a number of things have convinced me it is time to have a full time tool-guy at id.
The original QE4 editor was my very first ever Win32 program, and while Robert has done a good job of keeping it on life support through necessary extensions and evolutions, it really is past its designated lifespan.
I want to take a shot at making the level editor cross platform, so it can be used on linux and mac systems. I'm not exactly confident that it will work out, but I want to try.
Many of the content creation tasks seem to be fairly stabilized over the last several years, and our next product is going to be pretty content directed, so I can justify more engineering effort on writing better tools.
It is time for a re-think of the relationships between editor, utilities, and game. I am still an opponent of in-game editors, but I want to rearrange a lot of things so that some subsystems can be shared.
All of that added up to more than I was going to be able to do in the time left after the various research, graphics, and networking things I want to pursue.
* added r_speeds timing info to cinematic texture uploads
* fixed loop sounds on movers
* new bfg sound
* "sv_pure 1" as default, requires clients to only get data from pk3 files the server is using
* fixed fog pass on inside of textured fog surfaces
* properly fog sprites
* graphics for scoreboard headers
* show colored names on attacker display and scoreboard
* made "no such frame" a developer only warning
* count a disconnect while losing in tournement mode as a win for the other player
* fixed running with jump held down runs slow
* draw "connection problems" in center of screen in addition to phone jack icon
* cut marks properly on non-convex faces
* fixed bug with command completion partial match and case sensitivity
* fixed console dropping on level start
* fixed frags left feedback after restarts
* fog after dlight
* removed fogged stages from shader, dynamically generate
* removed fogonly shader keyword, infer from surfaceparm
* removed uneeded reinit of shader system on vid_restart
-----------------------------------------
John Carmack's .plan for Nov 01, 1999
-----------------------------------------
* play chat sound during votes
* draw 2D icon for ammo if cg_draw3dicons 0
* fixed losing input on menu vid_restart
* made "vote" and "callvote" completable
* remove mac about dialog
* fixed demos skipping inital time due to loading
* fixed timing on timedemo startup
* don't flash attacker when damaging self
* display capturelimit instead of fraglimit on score tabs in ctf
* recursive mirror/portal changed to a developer warning
* fixed bug with follow mode spectators
* battle suit shader
* notsingle spawn option
* separated torso and legs priority animation coutners so gesture doesn't mess with legs
* adjusted value to prevent missed launch sound on accelerator pads
* setviewpos x y z yaw, same parms as viewpos command
* stop sound on file access
* fixed developer prints from renderer
* defered client media loading, only load models and sounds for new players when you die or bring up the scoreboard
* fix for double colliding against same plane and getting stuck
* dropped LG damage to 160 pts / sec
* don't snap predicted player entity, smooths deaths and movers
* all sine movers are instant kill on block
* fixed items riding on bobbers
-----------------------------------------
John Carmack's .plan for Nov 02, 1999
-----------------------------------------
* remove grapple from give all
* fixed pick-up-two-healths-even-if-you-don't-need-the-second bug
* moved wrap/clamp to image generation function and added to imagelist, fixed an improper clamp on macs
* different menuback for ragepro
* fixed mac button problems with OS9 and wheel mice
* teamplay rule mods:
less MG damage (5 instead of 7)
weapons always have a full load of ammo
don't drop powerups
* changed low detail r_subdivisions to 25 to prevent poke through
* removed warning on empty servercommand when systeminfo changes
* g_debugDamage
* when a vote is tied with all votes in, immediately fail it
* haste smoke puffs
-----------------------------------------
John Carmack's .plan for Nov 03, 1999
-----------------------------------------
* cd check in single player
* removed drop shadow on console input line
* swapped mynx pain sounds
* fixed cleared music buffer on track loop again
* force skins on spectators in team games to prevent having a default waste memory
* only defer to a model with same team skin
* fixed grenade-disappearing-at-floor bug when about to explode
* draw reward medals during gameplay
* added "humiliation" feedback for gauntlet kills
* spread respawn times to prevent pattern running:
[code]
#define RESPAWN_ARMOR 25
#define RESPAWN_TEAM_WEAPON 30
#define RESPAWN_HEALTH 35
#define RESPAWN_AMMO 40
#define RESPAWN_HOLDABLE 60
#define RESPAWN_MEGAHEALTH 120
#define RESPAWN_POWERUP 120
[/code]
-----------------------------------------
John Carmack's .plan for Nov 05, 1999
-----------------------------------------
* check for bad weapon number in non-3d ammo icon on death
* fixed plane catagorization
* error command does an ERR_DROP if given a parm
* don't load high LOD models if r_lodbias
* nobots/nohumans options for player spawn spots
* prevent voice rewards on frag that enters intermission
* dissallow native dll loading if sv_pure
* loaddefered cgame command, issued on addbot
* drop the weapon you are changing TO if you only had a MG or gauntlet
* fixed bounce pad event prediction for all angles
* allow empty tokens in map files
* fixed infos exceeded warning on bot parse
* warning on mismatched mipmap/picmip/wrapclamp image reuse
* fixed pain echo sound after predicted falling damage
* move sound to hunk
* move vm to hunk
* restart game vm in place for map_restarts
* avoid all lightmaps entirely when using vertex light
* pretouch all images after registration
* pretouch all known in-use memory before starting a game
* on error, shutdown client before server, to be more likely to get out of fullscreen before a recursive error
* new pre-allocated memmory manager to crutch up mac
* meminfo command replaces hunk_stats and z_stats
* adjusted scoreboard titles
* no guantlet reward on corpses
* fixed snd_restart when paused
* PRE_RELEASE_DEMO hack
-----------------------------------------
John Carmack's .plan for Nov 09, 1999
-----------------------------------------
* fixed spinning barrel on respawn issue
* clear eflags before intermission
* shutdown menu on starting a cinematic
* mask name colors to 0-7 range
* fixed jpeg loading alpha channel
* try for not-nearest spawn twice instead of once
* made unzoomed exactly identity mouse modifier
* cl_debugmove [1/2]
* m_filter
* fixed time warnings
* allow timelimits to hit with only a single player
* filter local games with different protocol versions
* fixed bad arg 0 after sysinfo configstring
* removed unneeded svc_servercommand at start of command strings
* fixed redundantly loaded level bug
* fixed journal playback from demo build
* removed background image from viewlog window
-----------------------------------------
John Carmack's .plan for Nov 11, 1999
-----------------------------------------
* teamplay menu comment
* shrank and moved "RECORDING demo:" text
* identified and worked around Apple input queue issue
* properly send configstring resets on map_restart
* don't clip sound buffer on file writes
* don't draw scoreboard during warmup
* auto load added bots in single player
* swapped order of map_restart and warmup configstring
* disable dynamic lights on riva128 due to lack of blend mode
* put frags left warning back in all gametypes
* removed joystick button debug prints
-----------------------------------------
John Carmack's .plan for Nov 13, 1999
-----------------------------------------
* graphic for defer
* don't set any systeminfo vars from demos
* A3D fix
* spectator follow clients that disconnect
* stop follow mode before going to intermission so you can ready
* use (fullbright) vertex lighting if the bsp file doesn't have lightmaps
* auto set demo keyword on servers
* finished cd key authorization
* fixed symbol table loading for interpreter
* reconnect command
* removed limit on number of completed commands
* changed default name to "UnnamedPlayer"
* awards over people's heads in multiplayer
* fixed global powerup announcements
-----------------------------------------
John Carmack's .plan for Nov 14, 1999
-----------------------------------------
The demo test is built. It should go up in a couple hours if nothing explodes.
Mac and linux builds won't be out tonight.
* clear SVF_BOT when exiting follow mode
* render temp memory
* new mac GL initialization code
* no zone memory use in music thread
* added check for trash past zone block
* explicitly flush journal data file after a write
* added FS_Flush
-----------------------------------------
John Carmack's .plan for Nov 15, 1999
-----------------------------------------
The way vertex lighting is working in the existing demos is that only two pass shaders (lightmap * texture) were collapsed to a single pass, all other shaders stayed the same.
Xian added some chrome and energy effects to parts of q3tourney2, which changed them from two pass to three pass shaders. We felt that that 50% increase on those polygons was justified in normal play, but as people have pointed out, when you are playing with vertex lighting, that three passes stays three passes instead of collapsing to a single pass, resulting in a 300% increase on those polygons over the way it was before. Still faster than lightmap mode, but a large variance over other parts of the level.
Today I wrote new code to address that, and improve on top of it.
Now when r_vertexlight is on, I force every single shader to a single pass. In the cases where it isn't a simple light*texture case, I try and intelligently pick the most representative pass and do some fixups on the shader modulations.
This works our great, and brings the graphics load down to the minimum we can do with the data sets.
Performance is still going to be down a couple msec a frame due to using dynamic compilation instead of dll's for the cgame, but that is an intentional tradeoff. You can obviously slow things down by running a lot of bots, but that is to be expected.
I am still investigating the high idle dedicated server cpu utilization and a few other issues. The server cpu time will definately be higher than 1.08 due to the dynamic compiler, but again, that is an intentional tradeoff.
A set of go-fast-and-look-ugly options:
[code]
r_mode 2
r_colorbits 16
r_texturemode GL_LINEAR_MIPMAP_NEAREST
r_vertexlighting 1
r_subdivisions 999
r_lodbias 2
cg_gibs 0
cg_draw3dicons 0
cg_brassTime 0
cg_marks 0
cg_shadows 0
cg_simpleitems 1
cg_drawAttacker 0
[/code]
* icons for bot skills on scoreboard
* r_vertexlight is now "force single pass" for all shaders
* modified cd key check to be fire and forget on the client
* file handle debugging info in path command
* network address type of NA_BAD for failed resolves
* better command line variable overriding
* cache scoreboard for two seconds
* sync sound system before starting cinematics
* fixed many escapes disconnect from server exiting the game
* fixed shotgun pellets underwater expending all temp entities
-----------------------------------------
John Carmack's .plan for Nov 16, 1999
-----------------------------------------
The demo servers have general purpose flood-protection that has caused some confusion.
Clients are only allowed to make one command a second of any kind. This prevents excessive flooding by chats, model / name changes, and any other command that could possibly be exploited. The command streams are stalled, so it doesn't have any effect on processing order or reliability.
This means that if you issue two commands immediately after one another, there will be a one second stall before the second command and all movement clears. You see this on the lagometer as yellow spiking up for a second, then dropping away.
Hitting tab for the scoreboard sends a command, so you trigger the flood protection if you bang tab a couple times. This has been fixed so that the scoreboard will never send more than one update request every two seconds, but you will need to watch out for it in the existing demo.
The defered model loading has also caused some confusion, but that is a feature, not a bug. :)
In previous tests, you hitched for a second or two whenever a client with a new model or skin joined a game.
In the demo, when a client joins the game they will be given the model and skin of someone else temporarily, so there is no hitch. The only time it will hitch on entry is if it is a team game and there isn't anyone on the team they join. I make sure the skin color is correct, even if the model isn't.
These "defered" clients will be loaded when you bring up the scoreboard. You can do this directly by hitting tab, or you can have it happen for you when you die.
The point is: you died BEFORE it hitched, not as a result of the hitch.
The scoreboard header is up, but it is still a bit easy to miss.
* fixed high server idle cpu usage (it was spinning in place until maxfps was used!)
* fixed g_password, which is crashing in the demo
* moved svs.snapshotEntities to the hunk
* enable lagometer whenever running a non-local game
* cg_drawTeamOverlay cvar, set to 0 by default
* finished authorize work
* better reporting of unused highwater memory
-----------------------------------------
John Carmack's .plan for Nov 18, 1999
-----------------------------------------
The mac version is out. Go to www.quake3arena.com for links.
The mac version going out has the executable fixes that we have made in the last couple days, but most of the fixes have been in code that runs in the virtual machine, and we can't update that without making it incomptable with the pc version.
The game remains very marginal in playability on 266mhz imacs and iBooks.
A 333mhz imac should be playable for a casual gamer if the graphics options are turned down to the "fastest" setting.
There is still a lot of room for improvement on ATI's side with the RagePro drivers. Almost all the effort so far has been on the Rage128 drivers.
The G3 systems run fine, but a little slower than a pc of equal mhz
The rage128 cards in the G3s are only clocked at 75mhz, so you can't run too high of a resolution, but you can get very nice image quality. I usually play with these settings:
[code]
r_mode 2 // 512*284 res
r_colorbits 32 // 32 bit color
r_texturemode gl_linear_mipmap_linear // trilinear filtering
[/code]
I haven't played on one of the new iMacs or G4's but they both use the rage128 driver, which is fairly high quality now, so they should perform well.
We found a fairly significant problem with inputSprockets and mouse control (motion is dropped after 40msec). I have done a little working around it, so mouse control should be somewhat better in this version, but it will hopefully be fixed properly by Apple in the next IS rev. It isn't an issue if your framerate is high enough, but iMacs never see that framerate on their very best days..
Linux version tomorrow night, if nothing horrible happens.
Some advance warning about something that is sure to stir up some argument:
We should be handing off the masters for all three platforms within a day or two of each other, but they aren't going to show up in stores at the same time. Publishers, distributers, and stores are willing to go out of their way to expedite the arrival of the pc version, but they just won't go to the same amount of trouble for mac and linux boxes.
THE EXECUTABLES FOR ALL PLATFORMS WILL NOT BE AVAILABLE FOR DOWNLOAD UNTIL AFTER CHRISTMAS. This means that if you want to play on the mac or linux, don't pick up a copy of the pc version and expect to download the other executables.
Our first update to the game will be for all platforms, and will allow any version to be converted into any other, but we intend to hold that off for a little while.
We are doing this at the request of the distributors. The fear is that everyone will just grab a windows version, and the separate boxes will be ignored.
A lot of companies are going to be watching the sales figures for the mac and linux versions of Q3 to see if the platforms are actually worth supporting. If everyone bought a windows version and the other boxes sold like crap in comparison, that would be plenty of evidence for most executives to can any cross platform development.
I know there are a lot of people that play in both windows and linux, and this may be a bit of an inconvenience in the short term, but this is an ideal time to cast a vote as a consumer.
Its all the same to Id (I like hybrid CD's), and our continued support of linux and mac (OS X for the next title) is basically a foregone conclusion, but the results will probably influence other companies.
* fixed getting your own dropped / kicked message
* added developer print for all file open write's
* fixed occasional bad color on connecting background
* fixed occasional telefrag at start of skirmish game
* fix not being able to ready at intermission if you were following a bot
* never timelimit during tourney warmup
* fixed local timer on map_restart
* offset sorlag's head model for status bar
* added g_gametype to the votable commands: map, map_restart, kick, g_gametype
* changed sound registration sequence to fix losing default sound
* "sv_floodProtect 0" to turn off flood protection
* converted sequenced messages to static arrays
* fixed custom skin reassignment on all LOD
-----------------------------------------
John Carmack's .plan for Nov 19, 1999
-----------------------------------------
Linux version isn't going to make it tonight. We got too busy with other things. Sorry. Tomorrow.
* shrink zone, grow hunk
* flush memory on an error
* fixed crash pasting from clipboard
* test all compiler optimizations - 5% speedup
* fixed major slowdown in team games with large numbers of players and location markers
-----------------------------------------
John Carmack's .plan for Nov 21, 1999
-----------------------------------------
* fixed not-telefragging bug
* disabled flood protection with local clients
* fixed headoffset and gender on some model changes
* init cg_autoswitch cvars in cl
* fixed clearing of vm bss on restart
* fixed hang when looped part of song isn't found
* fixed two NAT clients connecting to same server
* fixed warning on random once only triggers
* added "g_allowVote 0"
* added developer background sound underrun warning
* move sound loading before clients so low memory defer works across maps
* changed cgame load failure to a drop
-----------------------------------------
John Carmack's .plan for Nov 22, 1999
-----------------------------------------
d:>mkdir research
Ahhhh...
I am very happy with how Q3 turned out. Probably more than any game we have done before, it's final form was very close to its initial envisioning.
I will be getting all the Q3 code we are going to release together over the next week or so. I will write some overview documentation to give a little context, and since you can do game mods without needing a commercial compiler now, I will write a brief step-by-step to modifying the game code.
I'm looking forward to what comes out of the community with Q3.
The rough outline of what I am going to be working on now:
We will be supporting Q3 for quite some time. Any problems we have will get fixed, and some new features my sneak in.
I have two rendering technologies that I intend to write research engines for.
I am going to spend some time on computer vision problems. I think the cheap little web cams have some interesting possibilities.
I am going to explore some possibilities with generalizing 3D game engines into more powerful environments with broader uses. I think that a lot of trends are coming to the point where a "cyberspace" as it is often imagined is begining to be feasible.
I am going to spend more time on some Free Software projects. I have been stealing a few hours here and there to work on the matrox glx project for a while now, and it has been pretty rewarding. People with an interest in the guts of a 3D driver might want to look at the project archives at http://glx.on.openprojects.net/. The web pages aren't very up to date, but the mailing list covers some good techie information.
-----------------------------------------
John Carmack's .plan for Dec 12, 1999
-----------------------------------------
WANTED: Independant OpenGL conformance nazi
I think there is a strong need for a proactive, vendor-neutral OpenGL watchdog, or even a small group, especially in the linux space.
I have been working on the utah-GLX team for quite a while now, and while I have been very pleased with the results, I would like to see more effort spent on doing things as right as possible. Because the developers (me included) are basically just doing the work in their spare time, testing usually only consists of running their favorite OpenGL application, and a few of the mesa demos, or some of the xscreensaver hacks.
Recently I did the initial bringup of a RagePro driver on linux, and I was much more conscious of the large untested feature space, and the tunnel vision I was using to get it to the point of running Q3.
What we need is someone, or a group of someones, who can really exercise different implementations through all corners of the OpenGL specification and provide detailed lists of faults with minimal test cases to reproduce the behavior.
In most cases, the bugs could then be fixed, but even if it is decided that the incorrect behavior is going to stay (to avoid a software fallback in a common accelerated case), there would be clear documentation of it.
I consider performance on the matrox driver right now to be "good enough". There is definately more performance oriented work going on, but given a choice of tasks to work on, I would rather improve quality and coverage instead of kicking a few more fps out of Q3.
One of Alex St. John's valid points was that "The drivers are always broken". There are a lot of factors that contribute to it, including fierce benchmarking competition causing driver writers to do some debatable things and diminish focus on quality. With open source drivers, some of those factors go away. Sure, it is nice to beat windows drivers on some benchmarks, but I wouldn't let pursuit of that goal introduce dumb things into the code.
Some of the windows IHVs have good testing proceedures and high quality drivers, but even there, it would be nice to have someone hounding them about things beyond how well quake releated games run.
The same goes for Apple, especially now that there is both ATI and 3dfx support.
Conformance would be my primary interest, but characterizing the performance of different drivers would also be usefull, especially for edge cases that may or may not be accelerated, like glDrawPixels.
On linux right now, we have:
The traditional fullscreen 3dfx mesa driver
The DRI-GLX based banshee/voodoo3 driver
The utah-GLX matrox G200/G400 driver
The temporary utah-GLX nvidia driver
The newly born utah-GLX ATI Rage Pro driver
If anyone is interested, join the developer list off of: http://glx.on.openprojects.net/
Doing a proper job would require a really good knowledge of the OpenGL specification, and a meticulous style, but it wouldn't require hardcore registers-and-dma driver writing skills, only basic glut programming.
If someone does wind up developing a good suite of tools and procedures and gives one of the drivers a really good set of feedback, I would be happy to provide extra video cards so they could beat up all the implementations.
-----------------------------------------
John Carmack's .plan for Dec 15, 1999
-----------------------------------------
Anna Kang left Id a couple weeks ago to found her own company - Fountainhead Entertainment.
It wasn't generally discussed during her time at Id, but we had been going out when she joined the company, and we were engaged earlier this year. We are getting married next month, and honeymooning in Hawaii. At her thoughtful suggestion, we are shipping a workstation out with us, so I don't fall into some programming-deprivation state. How great is that? :)
Now that Q3A has shipped, the official winner of her Id Software figurine chess set contest is Rowan Crawford for his prose and art.
An honorable mention goes to Reine Hogberg and Peder Hardings for their Q3A Blair Witch Project. They will receive silver Q3A medallions.
-----------------------------------------
John Carmack's .plan for Dec 21, 1999
-----------------------------------------
The Q3 game source code is getting pushed back a bit because we had to do some rearranging in the current codebase to facilitate the release, and we don't want to release in-progress code before the official binary point release.
We still have a Christmas present for the coders, though:
http://www.idsoftware.com/q1source/
Happy holidays!
-----------------------------------------
John Carmack's .plan for Dec 25, 1999
-----------------------------------------
There are a number of people upset about the Quake 1 source code release, because it is allowing cheating in existing games.
There will be a sorting out period as people figure out what directions the Quake1 world is going to go in with the new capabilities, but it will still be possible to have cheat free games after a few things get worked out.
Here's what needs to be done:
You have to assume the server is trusted. Because of the way quake mods work, It has always been possible to have server side cheats along the lines of "if name == mine, scale damage by 75%". You have to trust the server operator.
So, the problem then becomes a matter of making sure the clients are all playing with an acceptable version before allowing them to connect to the server. You obviously can't just ask the client, because if it is hacked it can just tell you what you want to hear. Because of the nature of the GPL, you can't just have a hidden part of the code to do verification.
What needs to be done is to create two closed source programs that act as executable loaders / verifiers and communication proxies for the client and server. These would need to be produced for each platform the game runs on. Some modifications will need to be done to the open source code to allow it to (optionally) communicate with these proxies.
These programs would perform a robust binary digest of the programs they are loading and communicate with their peer in a complex encrypted protocol before allowing the game connection to start. It may be possible to bypass the proxy for normal packets to avoid adding any scheduling or latency issues, but it will need to be involved to some degree to prevent a cheater from hijacking the connection once it is created.
The server operator would determine which versions of the game are to be allowed to connect to their server if they wish to enforce proxy protection. The part of the community that wants to be competetive will have to agree to some reasonable schedule of adoption of new versions.
Nothing in online games is cheat-proof (there is allways the device driver level of things to hack on), but that would actually be more secure than the game as it originally shipped, because hex edited patches wouldn't work any more. Someone could still in theory hack the closed source programs, but that is the same situation everyone was in with the original game.
People can start working on this immediately. There is some prior art in various unix games that would probably be helpfull. It would also be a good idea to find some crypto hackers to review proposed proxy communication strategies.
-----------------------------------------
John Carmack's .plan for Dec 29, 1999
-----------------------------------------
I have been playing a lot of Q3 on a 28.8 modem for the last several days.
I finally found a case of the stuck-at-awaiting-gamestate problem that turned out to be a continuous case of a fragment of the gamestate getting dropped. I have changed the net code to space out the sending of the fragments based on rate.
Note that there have been a few different things that result in stuck at gamestate or stuck at snapshot problems. We have fixed a few of them, but there may well still be other things that we haven't found yet.
You can still have a fun game on a 28.8 modem. It is a significant disadvantage, no question about it, but you can still have a good game if you play smart. If there is someone that knows what they are doing on a server with a ping in the low 100s, there won't usually be much you can do, but a skilled modem player can still beat up on unskilled T1 players..
Make sure your modem rate is set correctly. If you have it set too high, large amounts of data can get buffered up and you can wind up with multiple seconds of screwed up delays.
Only play on servers with good pings. My connection gives me a couple dozen servers with mid 200 pings. 56k modems often see servers with sub 200 pings. If you ignore the ping and just look for your favorite map, you will probably have a crappy game.
If you have a good basic connection to the server, the thing that will mess up your game is too much visible activity. This is a characteristic of the number of players, the openness of the level, and the weapons in use.
Don't play on madhouse levels with tons of players. None of the normal Q3 maps were really designed for more than eight players, and many were only designed for four.
Don't play in the wide open maps unless there are only a couple other players. Four very active players in a wide open area are enough to bog down a modem connection.
I just implemented "sv_minPing" / "sv_maxPing" options so servers can restrict themselves to only low ping or high ping players. This is done based on the ping of the challenge response packet, rather than any in-game pings. There are a few issues with that - a LPB may occasionally get into a HPB server if they happen to get a network hiccup at just the right time, and the number used as a gate will be closer to the number shown in the server list, rather than the number seen in gameplay. I would reccomend "sv_minPing 200" as a reasonable breakpoint.
-----------------------------------------
John Carmack's .plan for Dec 30, 1999
-----------------------------------------
Several people have mentioned an existing anti-cheat QW proxy that should also be applicable to modified versions:
http://www.students.tut.fi/~zibbo/qizmo/
-----------------------------------------
John Carmack's .plan for Feb 23, 2000
-----------------------------------------
This is a public statement that is also being sent directly to Slade at QuakeLives regarding http://www.quakelives.com/main/ql.cgi?section=dlagreement&file=qwcl-win32/
I see both sides of this. Your goals are positive, and I understand the issues and the difficulties that your project has to work under because of the GPL. I have also seen some GPL zealots acting petty and immature towards you very early on (while it is within everyone's rights to DEMAND code under the GPL, it isn't necessarily the best attitude to take), which probably colors some of your views on the subject.
We discussed several possible legal solutions to the issues.
This isn't one of them.
While I doubt your "give up your rights" click through would hold up in court, I am positive that you are required to give the source to anyone that asks for it that got a binary from someone else. This doesn't provide the obscurity needed for a gaming level of security.
I cut you a lot of slack because I honestly thought you intended to properly follow through with the requirements of the GPL, and you were just trying to get something fun out ASAP. It looks like I was wrong.
If you can't stand to work under the GPL, you should release the code to your last binary and give up your project. I would prefer that you continue your work, but abide by the GPL.
If necessary, I will pay whatever lawyer the Free Software Foundation reccomends to pursue this.
-----------------------------------------
John Carmack's .plan for Feb 24, 2000
-----------------------------------------
Some people took it upon themselves to remotely wreck Slade's development system. That is no more defensible than breaking into Id and smashing something.
The idea isn't to punish anyone, it is to have them comply with the license and continue to contribute. QuakeLives has quite a few happy users, and it is in everyone's best interest to have development continue. It just has to be by the rules.
-----------------------------------------
John Carmack's .plan for Mar 07, 2000
-----------------------------------------
Virtualized video card local memory is The Right Thing.
This is something I have been preaching for a couple years, but I finally got around to setting all the issues down in writing.
Now, the argument (and a whole bunch of tertiary information):
If you had all the texture density in the world, how much texture memory would be needed on each frame?
For directly viewed textures, mip mapping keeps the amount of referenced texels between one and one quarter of the drawn pixels. When anisotropic viewing angles and upper level clamping are taken into account, the number gets smaller. Take 1/3 as a conservative estimate.
Given a fairly aggressive six texture passes over the entire screen, that equates to needing twice as many texels as pixels. At 1024x768 resolution, well under two million texels will be referenced, no matter what the finest level of detail is. This is the worst case, assuming completely unique texturing with no repeating. More commonly, less than one million texels are actually needed.
As anyone who has tried to run certain Quake 3 levels in high quality texture mode on an eight or sixteen meg card knows, it doesn't work out that way in practice. There is a fixable part and some more fundamental parts to the fall-over-dead-with-too-many-textures problem.
The fixable part is that almost all drivers perform pure LRU (least recently used) memory management. This works correctly as long as the total amount of textures needed for a given frame fits in the card's memory after they have been loaded. As soon as you need a tiny bit more memory than fits on the card, you fall off of a performance cliff. If you need 14 megs of textures to render a frame, and your graphics card has 12 megs available after its frame buffers, you wind up loading 14 megs of texture data over the bus every frame, instead of just the 2 megs that don't fit. Having the cpu generate 14 megs of command traffic can drop you way into the single digit frame rates on most drivers.
If an application makes reasonable effort to group rendering by texture, and there is some degree of coherence in the order of texture references between frames, much better performance can be gotten with a swapping algorithm that changes its behavior instead of going into a full thrash:
[code]
While ( memory allocation for new texture fails )
Find the least recently used texture.
If the LRU texture was not needed in the previous frame,
Free it
Else
Free the most recently used texture that isn't bound to an active texture unit
[/code]
Freeing the MRU texture seems counterintuitive, but what it does is cause the driver to use the last bit of memory as a sort of scratchpad that gets constantly overwritten when there isn't enough space. Pure LRU plows over all the other textures that are very likely going to be needed at the beginning of the next frame, which will then plow over all the textures that were loaded on top of them.
If an application uses textures in a completely random order, any given replacement policy has the some effect..
Texture priority for swapping is a non-feature. There is NO benefit to attempting to statically prioritize textures for swapping. Either a texture is going to be referenced in the next frame, or it isn't. There aren't any useful gradations in between. The only hint that would be useful would be a notice that a given texture is not going to be in the next frame, and that just doesn't come up very often or cover very many texels.
With the MRU-on-thrash texture swapping policy, things degrade gracefully as the total amount of textures increase but due to several issues, the total amount of textures calculated and swapped is far larger than the actual amount of texels referenced to draw pixels.
The primary problem is that textures are loaded as a complete unit, from the smallest mip map level all the way up to potentially a 2048 by 2048 top level image. Even if you are only seeing 16 pixels of it off in the distance, the entire 12 meg stack might need to be loaded.
Packing can also cause some amount of wasted texture memory. When you want to load a two meg texture, it is likely going to require a lot more than just two megs of free texture memory, because a lot of it is going to be scattered around in 8k to 64k blocks. At the pathological limit, this can waste half your texture memory, but more reasonably it is only going to be 10% or so, and cause a few extra texture swap outs.
On a frame at a time basis, there are often significant amounts of texels even in referenced mip levels that are not seen. The back sides of characters, and large textures on floors can often have less than 50% of their texels used during a frame. This is only an issue as they are being swapped in, because they will very likely be needed within the next few frames. The result is one big hitch instead of a steady loading.
There are schemes that can help with these problems, but they have costs.
Packing losses can be addressed with compaction, but that has rarely proven to be worthwhile in the history of memory management. A 128-bit graphics accelerator could compact and sort 10 megs of texture memory in about 10 msec if desired.
The problems with large textures can be solved by just not using large textures. Both packing losses, and non- referenced texels can be reduced by chopping everything up into 64x64 or 128x128 textures. This requires preprocessing, adds geometry, and requires messy overlap of the textures to avoid seaming problems.
It is possible to estimate which mip levels will actually be needed and only swap those in. An application can't calculate exactly the mip map levels that will be referenced by the hardware, because there are slight variations between chips and the slope calculation would add significant processing overhead. A conservative upper bound can be taken by looking at the minimum normal distance of any vertex referencing a given texture in a frame. This will overestimate the required textures by 2x or so and still leave a big hit when the top mip level loads for big textures, but it can allow giant cathedral style scenes to render without swapping.
Clever programmers can always work harder to overcome obstacles, but in this case, there is a clear hardware solution that gives better performance than anything possible with software and just makes everyone's lives easier: virtualize the card's view of its local memory.
With page tables, address fragmentation isn't an issue, and with the graphics rasterizer only causing a page load when something from that exact 4k block is needed, the mip level problems and hidden texture problems just go away. Nothing sneaky has to be done by the application or driver, you just manage page indexes.
The hardware requirements are not very heavy. You need translation lookaside buffers (TLB) on the graphics chip, the ability to automatically load the TLB from a page table set up in local memory, and the ability to move a page from AGP or PCI into graphics memory and update the page tables and reference counts. You don't even need that many TLB, because graphics access patterns don't hop all over the place like CPU access can. Even with only a single TLB for each texture bilerp unit, reloads would only account for about 1/32 of the memory access if the textures were 4k blocked. All you would really want at the upper limit would be enough TLB for each texture unit to cover the texels referenced on a typical rasterization scan line.
Some programmers will say "I don't want the system to manage the textures, I want full control!" There are a couple responses to that. First, a page level management scheme has flexibility that you just can't get with a software only scheme, so it is a set of brand new capabilities. Second, you can still just choose to treat it as a fixed size texture buffer and manage everything yourself with updates. Third, even if it WAS slower than the craftiest possible software scheme (and I seriously doubt it), so much of development is about willingly trading theoretical efficiency for quicker, more robust development. We don't code overlays in assembly language any more..
Some hardware designers will say something along the lines of "But the graphics engine goes idle when you are pulling the page over from AGP!" Sure, you are always better off to just have enough texture memory and never swap, and this feature wouldn't let you claim any more megapixels or megatris, but every card winds up not having enough memory at some point. Ignoring those real world cases isn't helping your customers. In any case, it goes idle a hell of a lot less than if you were loading the entire texture over the command fifo.
3Dlabs is supposed to have some form of virtual memory management in the permedia 3, but I am not familiar with the details (if anyone from 3dlabs wants to send me the latest register specs, I would appreciate it!).
A mouse controlled first person shooter is fairly unique in how quickly it can change the texture composition of a scene. A 180-degree snap turn can conceivably bring in a completely different set of textures on a subsequent frame. Almost all other graphics applications bring textures in at a much steadier pace.
So, given that 180-degree snap turn to a completely different and uniquely textured scene, what would be the worst case performance? An AGP 2x bus is theoretically supposed to have over 500 mb/sec of bandwidth. It doesn't get that high in practice, but linear 4k block reads would give it the best possible conditions, and even at 300 mb/sec, reloading the entire texture working set would only take 10 msec.
Rendering is not likely to be buffered sufficiently to overlap appreciably with page loading, and the command transport for a complex scene will take significant time by itself, so it shows that a worst case scene will often not be able to be rendered in 1/60th of a second.
This is roughly the same lower bound that you get from a chip texturing directly from AGP memory. A direct AGP texture gains the benefit of fine-grained rendering overlap, but loses the benefit of subsequent references being in faster memory (outside of small on-chip caches). A direct AGP texture engine doesn't have the higher upper bounds of a cached texture engine, though. It's best and worst case are similar (generally a good thing), but the cached system can bring several times more bandwidth to bear when it isn't forced to swap anything in.
The important point is that the lower performance bound is almost an order of magnitude faster than swapping in the textures as a unit by the driver.
If you just positively couldn't deal with the chance of that much worst case delay, some form of mip level biasing could be made to kick in, or you could try and do pre-touching, but I don't think it would ever be worth it. The worst imaginable case is acceptable, and you just won't hit that case very often.
Unless a truly large number of TLB are provided, the textures would need to be blocked. The reason is that with a linear texture, a 4k page maps to only a couple scan lines on very large textures. If you are going with the grain you get great reuse, but if you go across it, you wind up referencing a new page every couple texel accesses. What is wanted is an addressing mechanism that converts a 4k page into a square area in the texture, so the page access is roughly constant for all orientations. There is also a benefit from having a 128 bit access also map to a square block of pixels, which several existing cards already do. The same interleaving-of-low-order-bits approach can just be extended a few more bits.
Dealing with blocked texture patterns is a hassle for a driver writer, but most graphics chips have a host blit capability that should let the chip deal with changing a linear blit into blocked writes. Application developers should never know about it, in any case.
There are some other interesting things that could be done if the page tables could trigger a cpu interrupt in addition to being automatically backed by AGP or PCI memory. Textures could be paged in directly from disk for truly huge settings, or decompressed from jpeg blocks, or even procedurally generated. Even the size limits of the AGP aperture could usefully be avoided if the driver wanted to manage each page's allocation.
Aside from all the basic swapping issue, there are a couple of other hardware trends that push things this way.
Embedded dram should be a driving force. It is possible to put several megs of extremely high bandwidth dram on a chip or die with a video controller, but won't be possible (for a while) to cram a 64 meg geforce in. With virtualized texturing, the major pressure on memory is drastically reduced. Even an 8mb card would be sufficient for 16 bit 1024x768 or 32 bit 800x600 gaming, no matter what the texture load.
The only thing that prevents a geometry processor based card from turning almost any set of commands in a display list into a single static dma buffer is the fact that textures may be swapped in and out, causing the register programming in the buffer to be wrong. With virtual texture addressing, a texture's address never changes, and an arbitrarily complex model can be described in a static dma buffer.
-----------------------------------------
John Carmack's .plan for Mar 27, 2000
-----------------------------------------
Seumas McNally
Two years ago, Id was contacted by the Startlight Foundation, an organization that tries to grant wishes to seriously ill kids. (www.starlight.org)
There was a young man with Hodgkin's Lymphoma that, instead of wanting to go to Disneyland or other traditional wishes, wanted to visit Id and talk with me about programming.
It turned out that Seumas McNally was already an accomplished developer. His family company, Longbow Digital Arts (www.longbowdigitalarts.com), had been doing quite respectably selling small games directly over the internet. It bore a strong resemblance to the early shareware days of Apogee and Id.
We spent the evening talking about graphics programmer things - the relative merits of voxels and triangles, procedurally generated media, level of detail management, API and platforms.
We talked at length about the balance between technology and design, and all the pitfalls that lie in the way of shipping a modern product.
We also took a dash out in my ferrari, thinking "this is going to be the best excuse a cop will ever hear if we get pulled over".
Longbow continued to be successful, and eventually the entire family was working full time on "Treadmarks", their new 3D tank game.
Over email about finishing the technology in Treadmarks, Seumas once said "I hope I can make it". Not "be a huge success" or "beat the competition". Just "make it".
That is a yardstick to measure oneself by.
It is all too easy to lose your focus or give up with just the ordinary distractions and disappointments that life brings. This wasn't ordinary. Seumas had cancer. Whatever problems you may be dealing with in your life, they pale before having problems drawing your next breath.
He made it.
Treadmarks started shipping a couple months ago, and was entered in the Independent Games Festival at the Game Developer's Conference this last month. It came away with the awards for technical excellence, game design, and the grand prize.
I went out to dinner with the McNally family the next day, and had the opportunity to introduce Anna to them. One of the projects at Anna's new company, Fountainhead Entertainment (www.fountainheadent.com), is a documentary covering gaming, and she had been looking forward to meeting Seumas after hearing me tell his story a few times. The McNallys invited her to bring a film crew up to Canada and talk with everyone whenever she could.
Seumas died the next week.
I am proud to have been considered an influence in Seumas' work, and I think his story should be a good example for others. Through talent and determination, he took something he loved and made a success out of it in many dimensions.
http://www.gamedev.net/community/memorial/seumas/ for more information.
-----------------------------------------
John Carmack's .plan for Apr 06, 2000
-----------------------------------------
Whenever I start a new graphics engine, I always spend a fair amount of time flipping back through older graphics books. It is always interesting to see how your changed perspective with new experience impacts your appreciation of a given article.
I was skimming through Jim Blinn's "A Trip Down The Graphics Pipeline" tonight, and I wound up laughing out loud twice.
From the book:
P73: I then empirically found that I had to scale by -1 in x instead of in z, and also to scale the xa and xf values by -1. (Basically I just put in enough minus signs after the fact to make it work.) Al Barr refers to this technique as "making sure you have made an even number of sign errors."
P131: The only lines that generate w=0 after clipping are those that pass through the z axis, the valley of the trough. These lines are lines that pass exactly through the eyepoint. After which you are dead and don't care about divide-by-zero errors.
If you laughed, you are a graphics geek.
My first recollection of a Jim Blinn article many years ago was my skimming over it and thinking "My god, what ridiculously picky minutia." Over the last couple years, I found myself haranguing people over some fairly picky issues, like the LSB errors with cpu vs rasterizer face culling and screen edge clipping with guard band bit tests. After one of those pitches, I quite distinctly thought to myself "My god, I'm turning into Jim Blinn!" :)
-----------------------------------------
John Carmack's .plan for Apr 29, 2000
-----------------------------------------
We need more bits per color component in our 3D accelerators.
I have been pushing for a couple more bits of range for several years now, but I now extend that to wanting full 16 bit floating point colors throughout the graphics pipeline. A sign bit, ten bits of mantissa, and five bits of exponent (possibly trading a bit or two between the mantissa and exponent). Even that isn't all you could want, but it is the rational step.
It is turning out that I need a destination alpha channel for a lot of the new rendering algorithms, so intermediate solutions like 10/12/10 RGB formats aren't a good idea. Higher internal precision with dithering to 32 bit pixels would have some benefit, but dithered intermediate results can easily start piling up the errors when passed over many times, as we have seen with 5/6/5 rendering.
Eight bits of precision isn't enough even for full range static image display. Images with a wide range usually come out fine, but restricted range images can easily show banding on a 24-bit display. Digital television specifies 10 bits of precision, and many printing operations are performed with 12 bits of precision.
The situation becomes much worse when you consider the losses after multiple operations. As a trivial case, consider having multiple lights on a wall, with their contribution to a pixel determined by a texture lookup. A single light will fall off towards 0 some distance away, and if it covers a large area, it will have visible bands as the light adds one unit, two units, etc. Each additional light from the same relative distance stacks its contribution on top of the earlier ones, which magnifies the amount of the step between bands: instead of going 0,1,2, it goes 0,2,4, etc. Pile a few lights up like this and look towards the dimmer area of the falloff, and you can believe you are back in 256-color land.
There are other more subtle issues, like the loss of potential result values from repeated squarings of input values, and clamping issues when you sum up multiple incident lights before modulating down by a material.
Range is even more clear cut. There are some values that have intrinsic ranges of 0.0 to 1.0, like factors of reflection and filtering. Normalized vectors have a range of -1.0 to 1.0. However, the most central quantity in rendering, light, is completely unbounded. We want a LOT more than a 0.0 to 1.0 range. Q3 hacks the gamma tables to sacrifice a bit of precision to get a 0.0 to 2.0 range, but I wanted more than that for even primitive rendering techniques. To accurately model the full human sensable range of light values, you would need more than even a five bit exponent.
This wasn't much of an issue even a year ago, when we were happy to just cover the screen a couple times at a high framerate, but realtime graphics is moving away from just "putting up wallpaper" to calculating complex illumination equations at each pixel. It is not at all unreasonable to consider having twenty textures contribute to the final value of a pixel. Range and precision matter.
A few common responses to this pitch:
"64 bits per pixel??? Are you crazy???" Remember, it is exactly the same relative step as we made from 16 bit to 32 bit, which didn't take all that long.
Yes, it will be slower. That's ok. This is an important point: we can't continue to usefully use vastly greater fill rate without an increase in precision. You can always crank the resolution and multisample anti-alaising up higher, but that starts to have diminishing returns well before you use of the couple gigatexels of fill rate we are expected to have next year. The cool and interesting things to do with all that fill rate involves many passes composited into less pixels, making precision important.
"Can we just put it in the texture combiners and leave the framebuffer at 32 bits?" No. There are always going to be shade trees that overflow a given number of texture units, and they are going to be the ones that need the extra precision. Scales and biases between the framebuffer and the higher precision internal calculations can get you some mileage (assuming you can bring the blend color into your combiners, which current cards can't), but its still not what you want. There are also passes which fundamentally aren't part of a single surface, but still combine to the same pixels, as with all forms of translucency, and many atmospheric effects.
"Do we need it in textures as well?" Not for most image textures, but it still needs to be supported for textures that are used as function look up tables.
"Do we need it in the front buffer?" Probably not. Going to a 64 bit front buffer would probably play hell with all sorts of other parts of the system. It is probably reasonable to stay with 32 bit front buffers with a blit from the 64 bit back buffer performing a lookup or scale and bias operation before dithering down to 32 bit. Dynamic light adaptation can also be done during this copy. Dithering can work quite well as long as you are only performing a single pass.
I used to be pitching this in an abstract "you probably should be doing this" form, but two significant things have happened that have moved this up my hit list to something that I am fairly positive about.
Mark Peercy of SGI has shown, quite surprisingly, that all Renderman surface shaders can be decomposed into multi-pass graphics operations if two extensions are provided over basic OpenGL: the existing pixel texture extension, which allows dependent texture lookups (matrox already supports a form of this, and most vendors will over the next year), and signed, floating point colors through the graphics pipeline. It also makes heavy use of the existing, but rarely optimized, copyTexSubImage2D functionality for temporaries.
This is a truly striking result. In retrospect, it seems obvious that with adds, multiplies, table lookups, and stencil tests that you can perform any computation, but most people were working under the assumption that there were fundamentally different limitations for "realtime" renderers vs offline renderers. It may take hundreds or thousands of passes, but it clearly defines an approach with no fundamental limits. This is very important. I am looking forward to his Siggraph paper this year.
Once I set down and started writing new renderers targeted at GeForce level performance, the precision issue has started to bite me personally. There are quite a few times where I have gotten visible banding after a set of passes, or have had to worry about ordering operations to avoid clamping. There is nothing like actually dealing with problems that were mostly theoretical before..
64 bit pixels. It is The Right Thing to do. Hardware vendors: don't you be the company that is the last to make the transition.
-----------------------------------------
John Carmack's .plan for May 08, 2000
-----------------------------------------
The .qc files for quake1/quakeworld are now available under the GPL in source/qw-qc.tar.gx on out ftp site. This was an oversight on my part in the original release.
Thanks to the QuakeForge team for doing the grunt work of the preparation.
-----------------------------------------
John Carmack's .plan for May 09, 2000
-----------------------------------------
And the Q1 utilities are now also available under the GPL in source/q1tools_gpl.tgz.
-----------------------------------------
John Carmack's .plan for May 14, 2000
-----------------------------------------
I stayed a couple days after E3 to attend the SORAC amateur rocket launch. I have provided some sponsorship to two of the teams competing for the CATS (Cheap Access to Space) rocketry prize, and it was a nice opportunity to get out and meet some of the people.
It is interesting how similar the activity is around an experimental rocket launch, going to a race track with an experimental car, and putting out a beta version of new software is. Lots of "twenty more minutes!", and lots of well-wishers waiting around while the people on the critical path sweat over what they are doing.
Mere minutes before we absolutely, positively needed to leave to catch our plane flight, they started the countdown. The rocket launched impressively, but broke apart at a relatively low altitude. Ouch. It was a hybrid, so there wasn't really an explosion, but watching the debris rain down wasn't very heartening. Times like that, I definitely appreciate working in software. "Run it again, with a breakpoint!"
Note to self: pasty-skinned programmers ought not stand out in the Mojave desert for multiple hours.
http://www.space-frontier.org/Events/CATSPRIZE_1/
http://www.energyrs.com/sorac/sorac.htm
http://www.jpaerospace.com/
-----------------------------------------
John Carmack's .plan for May 17, 2000
-----------------------------------------
I have gotten a lot of requests for comments on the latest crop of video cards, so here is my initial technical evaluation. We have played with some early versions, but this is a paper evaluation. I am not in a position to judge 2D GDI issues or TV/DVD issues, so this is just 3D commentary.
Nvidia Marketing silliness: saying "seven operations on a pixel" for a dual texture chip. Yes, I like NV_register_combiners a lot, but come on..
The DDR GeForce is the reigning champ of 3D cards. Of the shipping boards, it is basically better than everyone at every aspect of 3D graphics, and pioneered some features that are going to be very important: signed pixel math, dot product blending, and cubic environment maps.
The GeForce2 is just a speed bumped GeForce with a few tweaks, but that's not a bad thing. Nvidia will have far and away the tightest drivers for quite some time, and that often means more than a lot of new features in the real world.
The nvidia register combiners are highly programmable, and can often save a rendering pass or allow a somewhat higher quality calculation, but on the whole, I would take ATI's third texture for flexibility.
Nvidia will probably continue to hit the best framerates in benchmarks at low resolution, because they have flexible hardware with geometry acceleration and well-tuned drivers.
GeForce is my baseline for current rendering work, so I can wholeheartedly recommend it.
ATI Marketing silliness: "charisma engine" and "pixel tapestry" are silly names for vertex and pixel processing that are straightforward improvements over existing methods. Sony is probably to blame for starting that.
The Radeon has the best feature set available, with several advantages over GeForce:
A third texture unit per pixel
Three dimensional textures
Dependent texture reads (bump env map)
Greater internal color precision.
User clip planes orthogonal to all rasterization modes.
More powerful vertex blending operations.
The shadow id map support may be useful, but my work with shadow buffers have shown them to have significant limitations for global use in a game.
On paper, it is better than GeForce in almost every way except that it is limited to a maximum of two pixels per clock while GeForce can do four. This comes into play when the pixels don't do as much memory access, for example when just drawing shadow planes to the depth/stencil buffer, or when drawing in roughly front to back order and many of the later pixels depth fail, avoiding the color buffer writes.
Depending on the application and algorithm, this can be anywhere from basically no benefit when doing 32 bit blended multi-pass, dual texture rendering to nearly double the performance for 16 bit rendering with compressed textures. In any case, a similarly clocked GeForce(2) should somewhat outperform a Radeon on today's games when fill rate limited. Future games that do a significant number of rendering passes on the entire world may go back in ATI's favor if they can use the third texture unit, but I doubt it will be all that common.
The real issue is how quickly ATI can deliver fully clocked production boards, bring up stable drivers, and wring all the performance out of the hardware. This is a very different beast than the Rage128. I would definitely recommend waiting on some consumer reviews to check for teething problems before upgrading to a Radeon, but if things go well, ATI may give nvidia a serious run for their money this year.
3DFX Marketing silliness: Implying that a voodoo 5 is of a different class than a voodoo 4 isn't right. Voodoo 4 max / ultra / SLI / dual / quad or something would have been more forthright.
Rasterization feature wise, voodoo4 is just catching up to the original TNT. We finally have 32 bit color and stencil. Yeah.
There aren't any geometry features.
The T buffer is really nothing more than an accumulation buffer that is averaged together during video scanout. This same combining of separate buffers can be done by any modern graphics card if they are set up for it (although they will lose two bits of color precision in the process). At around 60 fps there is a slight performance win by doing it at video scannout time, but at 30 fps it is actually less memory traffic to do it explicitly. Video scan tricks also usually don't work in windowed modes.
The real unique feature of the voodoo5 is subpixel jittering during rasterization, which can't reasonably be emulated by other hardware. This does indeed improve the quality of anti-aliasing, although I think 3dfx might be pushing it a bit by saying their 4 sample jittering is as good as 16 sample unjittered.
The saving grace of the voodoo5 is the scalability. Because it only uses SDR ram, a dual chip Voodoo5 isn't all that much faster than some other single chip cards, but the quad chip card has over twice the pixel fill rate of the nearest competitor. That is a huge increment. Voodoo5 6000 should win every benchmark that becomes fill rate limited.
I haven't been able to honestly recommend a voodoo3 to people for a long time, unless they had a favorite glide game or wanted early linux Xfree 4.0 3D support. Now (well, soon), a Voodoo5 6000 should make all of today's games look better than any other card. You can get over twice as many pixel samples, and have them jittered and blended together for anti-aliasing.
It won't be able to hit Q3 frame rates as high as GeForce, but if you have a high end processor there really may not be all that much difference for you between 100fps and 80fps unless you are playing hardcore competitive and can't stand the occasional drop below 60fps.
There are two drawbacks: it's expensive, and it won't take advantage of the new rasterization features coming in future games. It probably wouldn't be wise to buy a voodoo5 if you plan on keeping it for two years.
-----------------------------------------
John Carmack's .plan for Jun 01, 2000
-----------------------------------------
Well, this is going to be an interesting .plan update.
Most of this is not really public business, but if some things aren't stated explicitly, it will reflect unfairly on someone.
As many people have heard discussed, there was quite a desire to remake DOOM as our next project after Q3. Discussing it brought an almost palpable thrill to most of the employees, but Adrian had a strong enough dislike for the idea that it was shot down over and over again.
Design work on an alternate game has been going on in parallel with the mission pack development and my research work.
Several factors, including a general lack of enthusiasm for the proposed plan, the warmth that Wolfenstien was met with at E3, and excitement about what we can do with the latest rendering technology were making it seem more and more like we weren't going down the right path.
I discussed it with some of the other guys, and we decided that it was important enough to drag the company through an unpleasant fight over it.
An ultimatum was issued to Kevin and Adrian(who control >50% of the company): We are working on DOOM for the next project unless you fire us.
Obviously no fun for anyone involved, but the project direction was changed, new hires have been expedited, and the design work has begun.
It wasn't planned to announce this soon, but here it is: We are working on a new DOOM game, focusing on the single player game experience, and using brand new technology in almost every aspect of it. That is all we are prepared to say about the game for quite some time, so don't push for interviews. We will talk about it when things are actually built, to avoid giving misleading comments.
It went smoother than expected, but the other shoe dropped yesterday.
Kevin and Adrian fired Paul Steed in retaliation, over my opposition.
Paul has certainly done things in the past that could be grounds for dismissal, but this was retaliatory for him being among the "conspirators".
I happen to think Paul was damn good at his job, and that he was going to be one of the most valuable contributors to DOOM.
We need to hire two new modeler/animator/cinematic director types. If you have a significant commercial track record in all three areas, and consider yourself at the top of your field, send your resume to Kevin Cloud.
-----------------------------------------
John Carmack's .plan for Feb 22, 2001
-----------------------------------------
GeForce 3 Overview
I just got back from Tokyo, where I demonstrated our new engine running under MacOS-X with a GeForce 3 card. We had quite a bit of discussion about whether we should be showing anything at all, considering how far away we are from having a title on the shelves, so we probably aren't going to be showing it anywhere else for quite a while.
We do run a bit better on a high end wintel system, but the Apple performance is still quite good, especially considering the short amount of time that the drivers had before the event.
It is still our intention to have a simultaneous release of the next product on Windows, MacOS-X, and Linux.
Here is a dump on the GeForce 3 that I have been seriously working with for a few weeks now:
The short answer is that the GeForce 3 is fantastic. I haven't had such an impression of raising the performance bar since the Voodoo 2 came out, and there are a ton of new features for programmers to play with.
Graphics programmers should run out and get one at the earliest possible time. For consumers, it will be a tougher call. There aren't any applications our right now that take proper advantage of it, but you should still be quite a bit faster at everything than GF2, especially with anti-aliasing. Balance that against whatever the price turns out to be.
While the Radeon is a good effort in many ways, it has enough shortfalls that I still generally call the GeForce 2 ultra the best card you can buy right now, so Nvidia is basically dethroning their own product.
It is somewhat unfortunate that it is labeled GeForce 3, because GeForce 2 was just a speed bump of GeForce, while GF3 is a major architectural change. I wish they had called the GF2 something else.
The things that are good about it:
Lots of values have additional internal precision, like texture coordinates and rasterization coordinates. There are only a few places where this matters, but it is nice to be cleaning up. Rasterization precision is about the last thing that the multi-thousand dollar workstation boards still do any better than the consumer cards.
Adding more texture units and more register combiners is an obvious evolutionary step.
An interesting technical aside: when I first changed something I was doing with five single or dual texture passes on a GF to something that only took two quad texture passes on a GF3, I got a surprisingly modest speedup. It turned out that the texture filtering and bandwidth was the dominant factor, not the frame buffer traffic that was saved with more texture units. When I turned off anisotropic filtering and used compressed textures, the GF3 version became twice as fast.
The 8x anisotropic filtering looks really nice, but it has a 30%+ speed cost. For existing games where you have speed to burn, it is probably a nice thing to force on, but it is a bit much for me to enable on the current project. Radeon supports 16x aniso at a smaller speed cost, but not in conjunction with trilinear, and something is broken in the chip that makes the filtering jump around with triangular rasterization dependencies.
The depth buffer optimizations are similar to what the Radeon provides, giving almost everything some measure of speedup, and larger ones available in some cases with some redesign.
3D textures are implemented with the full, complete generality. Radeon offers 3D textures, but without mip mapping and in a non-orthogonal manner (taking up two texture units).
Vertex programs are probably the most radical new feature, and, unlike most "radical new features", actually turn out to be pretty damn good. The instruction language is clear and obvious, with wonderful features like free arbitrary swizzle and negate on each operand, and the obvious things you want for graphics like dot product instructions.
The vertex program instructions are what SSE should have been.
A complex setup for a four-texture rendering pass is way easier to understand with a vertex program than with a ton of texgen/texture matrix calls, and it lets you do things that you just couldn't do hardware accelerated at all before. Changing the model from fixed function data like normals, colors, and texcoords to generalized attributes is very important for future progress.
Here, I think Microsoft and DX8 are providing a very good benefit by forcing a single vertex program interface down all the hardware vendor's throats.
This one is truly stunning: the drivers just worked for all the new features that I tried. I have tested a lot of pre-production 3D cards, and it has never been this smooth.
The things that are indifferent:
I'm still not a big believer in hardware accelerated curve tessellation. I'm not going to go over all the reasons again, but I would have rather seen the features left off and ended up with a cheaper part.
The shadow map support is good to get in, but I am still unconvinced that a fully general engine can be produced with acceptable quality using shadow maps for point lights. I spent a while working with shadow buffers last year, and I couldn't get satisfactory results. I will revisit that work now that I have GeForce 3 cards, and directly compare it with my current approach.
At high triangle rates, the index bandwidth can get to be a significant thing. Other cards that allow static index buffers as well as static vertex buffers will have situations where they provide higher application speed. Still, we do get great throughput on the GF3 using vertex array range and glDrawElements.
The things that are bad about it:
Vertex programs aren't invariant with the fixed function geometry paths. That means that you can't mix vertex program passes with normal passes in a multipass algorithm. This is annoying, and shouldn't have happened.
Now we come to the pixel shaders, where I have the most serious issues. I can just ignore this most of the time, but the way the pixel shader functionality turned out is painfully limited, and not what it should have been.
DX8 tries to pretend that pixel shaders live on hardware that is a lot more general than the reality.
Nvidia's OpenGL extensions expose things much more the way they actually are: the existing register combiners functionality extended to eight stages with a couple tweaks, and the texture lookup engine is configurable to interact between textures in a list of specific ways.
I'm sure it started out as a better design, but it apparently got cut and cut until it really looks like the old BumpEnvMap feature writ large: it does a few specific special effects that were deemed important, at the expense of a properly general solution.
Yes, it does full bumpy cubic environment mapping, but you still can't just do some math ops and look the result up in a texture. I was disappointed on this count with the Radeon as well, which was just slightly too hardwired to the DX BumpEnvMap capabilities to allow more general dependent texture use.
Enshrining the capabilities of this mess in DX8 sucks. Other companies had potentially better approaches, but they are now forced to dumb them down to the level of the GF3 for the sake of compatibility. Hopefully we can still see some of the extra flexibility in OpenGL extensions.
The future:
I think things are going to really clean up in the next couple years. All of my advocacy is focused on making sure that there will be a completely clean and flexible interface for me to target in the engine after DOOM, and I think it is going to happen.
The market may have shrunk to just ATI and Nvidia as significant players. Matrox, 3D labs, or one of the dormant companies may surprise us all, but the pace is pretty frantic.
I think I would be a little more comfortable if there was a third major player competing, but I can't fault Nvidia's path to success.
-----------------------------------------
John Carmack's .plan for Nov 16, 2001
-----------------------------------------
Driver optimizations have been discussed a lot lately because of the quake3 name checking in ATI's recent drivers, so I am going to lay out my position on the subject.
There are many driver optimizations that are pure improvements in all cases, with no negative effects. The difficult decisions come up when it comes to "trades" of various kinds, where a change will give an increase in performance, but at a cost.
Relative performance trades. Part of being a driver writer is being able to say "I don't care if stippled, anti-aliased points with texturing go slow", and optimizing accordingly. Some hardware features, like caches and hierarchical buffers, may be advantages on some apps, and disadvantages on others. Command buffer sizes often tune differently for different applications.
Quality trades. There is a small amount of wiggle room in the specs for pixel level variability, and some performance gains can be had by leaning towards the minimums. Most quality trades would actually be conformance trades, because the results are not exactly conformant, but they still do "roughly" the right thing from a visual standpoint. Compressing textures automatically, avoiding blending of very faint transparent pixels, using a 16 bit depth buffer, etc. A good application will allow the user to make most of these choices directly, but there is good call for having driver preference panels to enable these types of changes on naive applications. Many drivers now allow you to quality trade in an opposite manner - slowing application performance by turning on anti-aliasing or anisotropic texture filtering.
Conformance trades. Most conformance trades that happen with drivers are unintentional, where the slower, more general fallback case just didn't get called when it was supposed to, because the driver didn't check for a certain combination to exit some specially optimized path. However, there are optimizations that can give performance improvements in ways that make it impossible to remain conformant. For example, a driver could choose to skip storing of a color value before it is passed on to the hardware, which would save a few cycles, but make it impossible to correctly answer glGetFloatv( GL_CURRENT_COLOR, buffer ).
Normally, driver writers will just pick their priorities and make the trades, but sometimes there will be a desire to make different trades in different circumstances, so as to get the best of both worlds.
Explicit application hints are a nice way to offer different performance characteristics, but that requires cooperation from the application, so it doesn't help in an ongoing benchmark battle. OpenGL's glHint() call is the right thought, but not really set up as flexibly as you would like. Explicit extensions are probably the right way to expose performance trades, but it isn't clear to me that any conformant trade will be a big enough difference to add code for.
End-user selectable optimizations. Put a selection option in the driver properties window to allow the user to choose which application class they would like to be favored in some way. This has been done many times, and is a reasonable way to do things. Most users would never touch the setting, so some applications may be slightly faster or slower than in their "optimal benchmark mode".
Attempt to guess the application from app names, window strings, etc. Drivers are sometimes forced to do this to work around bugs in established software, and occasionally they will try to use this as a cue for certain optimizations.
My positions:
Making any automatic optimization based on a benchmark name is wrong. It subverts the purpose of benchmarking, which is to gauge how a similar class of applications will perform on a tested configuration, not just how the single application chosen as representative performs.
It is never acceptable to have the driver automatically make a conformance tradeoff, even if they are positive that it won't make any difference. The reason is that applications evolve, and there is no guarantee that a future release won't have different assumptions, causing the upgrade to misbehave. We have seen this in practice with Quake3 and derivatives, where vendors assumed something about what may or may not be enabled during a compiled vertex array call. Most of these are just mistakes, or, occasionally, laziness.
Allowing a driver to present a non-conformant option for the user to select is an interesting question. I know that as a developer, I would get hate mail from users when a point release breaks on their whiz-bang optimized driver, just like I do with overclocked CPUs, and I would get the same "but it works with everything else!" response when I tell them to put it back to normal. On the other hand, being able to tweak around with that sort of think is fun for technically inclined users. I lean towards frowning on it, because it is a slippery slope from there down in to "cheating drivers" of the see-through- walls variety.
Quality trades are here to stay, with anti-aliasing, anisotropic texture filtering, and other options being positive trades that a user can make, and allowing various texture memory optimizations can be a very nice thing for a user trying to get some games to work well. However, it is still important that it start from a completely conformant state by default. This is one area where application naming can be used reasonably by the driver, to maintain user selected per-application modifiers.
I'm not fanatical on any of this, because the overriding purpose of software is to be useful, rather than correct, but the days of game-specific mini- drivers that can just barely cut it are past, and we should demand more from the remaining vendors.
Also, excessive optimization is the cause of quite a bit of ill user experience with computers. Byzantine code paths extract costs as long as they exist, not just as they are written.
-----------------------------------------
John Carmack's .plan for Dec 21, 2001
-----------------------------------------
The Quake 2 source code is now available for download, licensed under the GPL.
ftp://ftp.idsoftware.com/idstuff/source/quake2.zip
As with previous source code releases, the game data remains under the original copyright and license, and cannot be freely distributed. If you create a true total conversion, you can give (or sell) a complete package away, as long as you abide by the GPL source code license. If your projects use the original Quake 2 media, the media must come from a normal, purchased copy of the game.
I'm sure I will catch some flack about increased cheating after the source release, but there are plenty of Q2 cheats already out there, so you are already in the position of having to trust the other players to a degree. The problem is really only solvable by relying on the community to police itself, because it is a fundamentally unwinnable technical battle to make a completely cheat proof game of this type. Play with your friends.
-----------------------------------------
John Carmack's .plan for Feb 11, 2002
-----------------------------------------
Last month I wrote the Radeon 8500 support for Doom.
The bottom line is that it will be a fine card for the game, but the details are sort of interesting.
I had a pre-production board before Siggraph last year, and we were discussing the possibility of letting ATI show a Doom demo behind closed doors on it. We were all very busy at the time, but I took a shot at bringing up support over a weekend. I hadn't coded any of the support for the custom ATI extensions yet, but I ran the game using only standard OpenGL calls (this is not a supported path, because without bump mapping everything looks horrible) to see how it would do. It didn't even draw the console correctly, because they had driver bugs with texGen. I thought the odds were very long against having all the new, untested extensions working properly, so I pushed off working on it until they had revved the drivers a few more times.
My judgment was colored by the experience of bringing up Doom on the original Radeon card a year earlier, which involved chasing a lot of driver bugs. Note that ATI was very responsive, working closely with me on it, and we were able to get everything resolved, but I still had no expectation that things would work correctly the first time.
Nvidia's OpenGL drivers are my "gold standard", and it has been quite a while since I have had to report a problem to them, and even their brand new extensions work as documented the first time I try them. When I have a problem on an Nvidia, I assume that it is my fault. With anyone else's drivers, I assume it is their fault. This has turned out correct almost all the time. I have heard more anecdotal reports of instability on some systems with Nivida drivers recently, but I track stability separately from correctness, because it can be influenced by so many outside factors.
ATI had been patiently pestering me about support for a few months, so last month I finally took another stab at it. The standard OpenGL path worked flawlessly, so I set about taking advantage of all the 8500 specific features. As expected, I did run into more driver bugs, but ATI got me fixes rapidly, and we soon had everything working properly. It is interesting to contrast the Nvidia and ATI functionality:
The vertex program extensions provide almost the same functionality. The ATI hardware is a little bit more capable, but not in any way that I care about. The ATI extension interface is massively more painful to use than the text parsing interface from nvidia. On the plus side, the ATI vertex programs are invariant with the normal OpenGL vertex processing, which allowed me to reuse a bunch of code. The Nvidia vertex programs can't be used in multipass algorithms with standard OpenGL passes, because they generate tiny differences in depth values, forcing you to implement EVERYTHING with vertex programs. Nvidia is planning on making this optional in the future, at a slight speed cost.
I have mixed feelings about the vertex object / vertex array range extensions. ATI's extension seems more "right" in that it automatically handles synchronization by default, and could be implemented as a wire protocol, but there are advantages to the VAR extension being simply a hint. It is easy to have a VAR program just fall back to normal virtual memory by not setting the hint and using malloc, but ATI's extension requires different function calls for using vertex objects and normal vertex arrays.
The fragment level processing is clearly way better on the 8500 than on the Nvidia products, including the latest GF4. You have six individual textures, but you can access the textures twice, giving up to eleven possible texture accesses in a single pass, and the dependent texture operation is much more sensible. This wound up being a perfect fit for Doom, because the standard path could be implemented with six unique textures, but required one texture (a normalization cube map) to be accessed twice. The vast majority of Doom light / surface interaction rendering will be a single pass on the 8500, in contrast to two or three passes, depending on the number of color components in a light, for GF3/GF4 (*note GF4 bitching later on).
Initial performance testing was interesting. I set up three extreme cases to exercise different characteristics:
A test of the non-textured stencil shadow speed showed a GF3 about 20% faster than the 8500. I believe that Nvidia has a slightly higher performance memory architecture.
A test of light interaction speed initially had the 8500 significantly slower than the GF3, which was shocking due to the difference in pass count. ATI identified some driver issues, and the speed came around so that the 8500 was faster in all combinations of texture attributes, in some cases 30+% more. This was about what I expected, given the large savings in memory traffic by doing everything in a single pass.
A high polygon count scene that was more representative of real game graphics under heavy load gave a surprising result. I was expecting ATI to clobber Nvidia here due to the much lower triangle count and MUCH lower state change functional overhead from the single pass interaction rendering, but they came out slower. ATI has identified an issue that is likely causing the unexpected performance, but it may not be something that can be worked around on current hardware.
I can set up scenes and parameters where either card can win, but I think that current Nvidia cards are still a somewhat safer bet for consistent performance and quality.
On the topic of current Nvidia cards:
Do not buy a GeForce4-MX for Doom.
Nvidia has really made a mess of the naming conventions here. I always thought it was bad enough that GF2 was just a speed bumped GF1, while GF3 had significant architectural improvements over GF2. I expected GF4 to be the speed bumped GF3, but calling the NV17 GF4-MX really sucks.
GF4-MX will still run Doom properly, but it will be using the NV10 codepath with only two texture units and no vertex shaders. A GF3 or 8500 will be much better performers. The GF4-MX may still be the card of choice for many people depending on pricing, especially considering that many games won't use four textures and vertex programs, but damn, I wish they had named it something else.
As usual, there will be better cards available from both Nvidia and ATI by the time we ship the game.
8:50 pm addendum: Mark Kilgard at Nvidia said that the current drivers already support the vertex program option to be invarint with the fixed function path, and that it turned out to be one instruction FASTER, not slower.
-----------------------------------------
John Carmack's .plan for Mar 15, 2002
-----------------------------------------
Mark Kilgard and Cass Everitt at Nvidia have released a paper on shadow volume rendering with several interesting bits in it. They also include a small document that I wrote a couple years ago about my discovery process during the development of some of the early Doom technology.
http://developer.nvidia.com/view.asp?IO=robust_shadow_volumes
-----------------------------------------
John Carmack's .plan for Jun 25, 2002
-----------------------------------------
The Matrox Parhelia Report:
The executive summary is that the Parhelia will run Doom, but it is not performance competitive with Nvidia or ATI.
Driver issue remain, so it is not perfect yet, but I am confident that Matrox will resolve them.
The performance was really disappointing for the first 256 bit DDR card. I tried to set up a "poster child" case that would stress the memory subsystem above and beyond any driver or triangle level inefficiencies, but I was unable to get it to ever approach the performance of a GF4.
The basic hardware support is good, with fragment flexibility better than GF4 (but not as good as ATI 8500), but it just doesn't keep up in raw performance. With a die shrink, this chip could probably be a contender, but there are probably going to be other chips out by then that will completely eclipse this generation of products.
None of the special features will be really useful for Doom:
The 10 bit color framebuffer is nice, but Doom needs more than 2 bits of destination alpha when a card only has four texture units, so we can't use it.
Anti aliasing features are nice, but it isn't all that fast in minimum feature mode, so nobody is going to be turning on AA. The same goes for "surround gaming". While the framerate wouldn't be 1/3 the base, it would still probably be cut in half.
Displacement mapping. Sigh. I am disappointed that the industry is still pursuing any quad based approaches. Haven't we learned from the stellar success of 3DO, Saturn, and NV1 that quads really suck? In any case, we can't use any geometry amplification scheme (including ATI's truform) in conjunction with stencil shadow volumes.
-----------------------------------------
John Carmack's .plan for Jun 27, 2002
-----------------------------------------
More graphics card notes:
I need to apologize to Matrox - their implementation of hardware displacement mapping is NOT quad based. I was thinking about a certain other companies proposed approach. Matrox's implementation actually looks quite good, so even if we don't use it because of the geometry amplification issues, I think it will serve the noble purpose of killing dead any proposal to implement a quad based solution.
I got a 3Dlabs P10 card in last week, and yesterday I put it through its paces. Because my time is fairly over committed, first impressions often determine how much work I devote to a given card. I didn't speak to ATI for months after they gave me a beta 8500 board last year with drivers that rendered the console incorrectly. :)
I was duly impressed when the P10 just popped right up with full functional support for both the fallback ARB_ extension path (without specular highlights), and the NV10 NVidia register combiners path. I only saw two issues that were at all incorrect in any of our data, and one of them is debatable. They don't support NV_vertex_program_1_1, which I use for the NV20 path, and when I hacked my programs back to 1.0 support for testing, an issue did show up, but still, this is the best showing from a new board from any company other than Nvidia.
It is too early to tell what the performance is going to be like, because they don't yet support a vertex object extension, so the CPU is hand feeding all the vertex data to the card at the moment. It was faster than I expected for those circumstances.
Given the good first impression, I was willing to go ahead and write a new back end that would let the card do the entire Doom interaction rendering in a single pass. The most expedient sounding option was to just use the Nvidia extensions that they implement, NV_vertex_program and NV_register_combiners, with seven texture units instead of the four available on GF3/GF4. Instead, I decided to try using the prototype OpenGL 2.0 extensions they provide.
The implementation went very smoothly, but I did run into the limits of their current prototype compiler before the full feature set could be implemented. I like it a lot. I am really looking forward to doing research work with this programming model after the compiler matures a bit. While the shading languages are the most critical aspects, and can be broken out as extensions to current OpenGL, there are a lot of other subtle-but-important things that are addressed in the full OpenGL 2.0 proposal.
I am now committed to supporting an OpenGL 2.0 renderer for Doom through all the spec evolutions. If anything, I have been somewhat remiss in not pushing the issues as hard as I could with all the vendors. Now really is the critical time to start nailing things down, and the decisions may stay with us for ten years.
A GL2 driver won't give any theoretical advantage over the current back ends optimized for cards with 7+ texture capability, but future research work will almost certainly be moving away from the lower level coding practices, and if some new vendor pops up (say, Rendition back from the dead) with a next-gen card, I would strongly urge them to implement GL2 instead of proprietary extensions.
I have not done a detailed comparison with Cg. There are a half dozen C-like graphics languages floating around, and honestly, I don't think there is a hell of a lot of usability difference between them at the syntax level. They are all a whole lot better than the current interfaces we are using, so I hope syntax quibbles don't get too religious. It won't be too long before all real work is done in one of these, and developers that stick with the lower level interfaces will be regarded like people that write all-assembly PC applications today. (I get some amusement from the all-assembly crowd, and it can be impressive, but it is certainly not effective)
I do need to get up on a soapbox for a long discourse about why the upcoming high level languages MUST NOT have fixed, queried resource limits if they are going to reach their full potential. I will go into a lot of detail when I get a chance, but drivers must have the right and responsibility to multipass arbitrarily complex inputs to hardware with smaller limits. Get over it.
-----------------------------------------
John Carmack's .plan for Jan 29, 2003
-----------------------------------------
NV30 vs R300, current developments, etc
At the moment, the NV30 is slightly faster on most scenes in Doom than the R300, but I can still find some scenes where the R300 pulls a little bit ahead. The issue is complicated because of the different ways the cards can choose to run the game.
The R300 can run Doom in three different modes: ARB (minimum extensions, no specular highlights, no vertex programs), R200 (full featured, almost always single pass interaction rendering), ARB2 (floating point fragment shaders, minor quality improvements, always single pass).
The NV30 can run DOOM in five different modes: ARB, NV10 (full featured, five rendering passes, no vertex programs), NV20 (full featured, two or three rendering passes), NV30 ( full featured, single pass), and ARB2.
The R200 path has a slight speed advantage over the ARB2 path on the R300, but only by a small margin, so it defaults to using the ARB2 path for the quality improvements. The NV30 runs the ARB2 path MUCH slower than the NV30 path. Half the speed at the moment. This is unfortunate, because when you do an exact, apples-to-apples comparison using exactly the same API, the R300 looks twice as fast, but when you use the vendor-specific paths, the NV30 wins.
The reason for this is that ATI does everything at high precision all the time, while Nvidia internally supports three different precisions with different performances. To make it even more complicated, the exact precision that ATI uses is in between the floating point precisions offered by Nvidia, so when Nvidia runs fragment programs, they are at a higher precision than ATI's, which is some justification for the slower speed. Nvidia assures me that there is a lot of room for improving the fragment program performance with improved driver compiler technology.
The current NV30 cards do have some other disadvantages: They take up two slots, and when the cooling fan fires up they are VERY LOUD. I'm not usually one to care about fan noise, but the NV30 does annoy me.
I am using an NV30 in my primary work system now, largely so I can test more of the rendering paths on one system, and because I feel Nvidia still has somewhat better driver quality (ATI continues to improve, though). For a typical consumer, I don't think the decision is at all clear cut at the moment.
For developers doing forward looking work, there is a different tradeoff - the NV30 runs fragment programs much slower, but it has a huge maximum instruction count. I have bumped into program limits on the R300 already.
As always, better cards are coming soon.
-----
Doom has dropped support for vendor-specific vertex programs (NV_vertex_program and EXT_vertex_shader), in favor of using ARB_vertex_program for all rendering paths. This has been a pleasant thing to do, and both ATI and Nvidia supported the move. The standardization process for ARB_vertex_program was pretty drawn out and arduous, but in the end, it is a just-plain-better API than either of the vendor specific ones that it replaced. I fretted for a while over whether I should leave in support for the older APIs for broader driver compatibility, but the final decision was that we are going to require a modern driver for the game to run in the advanced modes. Older drivers can still fall back to either the ARB or NV10 paths.
The newly-ratified ARB_vertex_buffer_object extension will probably let me do the same thing for NV_vertex_array_range and ATI_vertex_array_object.
Reasonable arguments can be made for and against the OpenGL or Direct-X style of API evolution. With vendor extensions, you get immediate access to new functionality, but then there is often a period of squabbling about exact feature support from different vendors before an industry standard settles down. With central planning, you can have "phasing problems" between hardware and software releases, and there is a real danger of bad decisions hampering the entire industry, but enforced commonality does make life easier for developers. Trying to keep boneheaded-ideas-that-will-haunt-us-for-years out of Direct-X is the primary reason I have been attending the Windows Graphics Summit for the past three years, even though I still code for OpenGL.
The most significant functionality in the new crop of cards is the truly flexible fragment programming, as exposed with ARB_fragment_program. Moving from the "switches and dials" style of discrete functional graphics programming to generally flexible programming with indirection and high precision is what is going to enable the next major step in graphics engines.
It is going to require fairly deep, non-backwards-compatible modifications to an engine to take real advantage of the new features, but working with ARB_fragment_program is really a lot of fun, so I have added a few little tweaks to the current codebase on the ARB2 path:
High dynamic color ranges are supported internally, rather than with post-blending. This gives a few more bits of color precision in the final image, but it isn't something that you really notice.
Per-pixel environment mapping, rather than per-vertex. This fixes a pet-peeve of mine, which is large panes of environment mapped glass that aren't tessellated enough, giving that awful warping-around-the-triangulation effect as you move past them.
Light and view vectors normalized with math, rather than a cube map. On future hardware this will likely be a performance improvement due to the decrease in bandwidth, but current hardware has the computation and bandwidth balanced such that it is pretty much a wash. What it does (in conjunction with floating point math) give you is a perfectly smooth specular highlight, instead of the pixelish blob that we get on older generations of cards.
There are some more things I am playing around with, that will probably remain in the engine as novelties, but not supported features:
Per-pixel reflection vector calculations for specular, instead of an interpolated half-angle. The only remaining effect that has any visual dependency on the underlying geometry is the shape of the specular highlight. Ideally, you want the same final image for a surface regardless of if it is two giant triangles, or a mesh of 1024 triangles. This will not be true if any calculation done at a vertex involves anything other than linear math operations. The specular half-angle calculation involves normalizations, so the interpolation across triangles on a surface will be dependent on exactly where the vertexes are located. The most visible end result of this is that on large, flat, shiny surfaces where you expect a clean highlight circle moving across it, you wind up with a highlight that distorts into an L shape around the triangulation line.
The extra instructions to implement this did have a noticeable performance hit, and I was a little surprised to see that the highlights not only stabilized in shape, but also sharpened up quite a bit, changing the scene more than I expected. This probably isn't a good tradeoff today for a gamer, but it is nice for any kind of high-fidelity rendering.
Renormalization of surface normal map samples makes significant quality improvements in magnified textures, turning tight, blurred corners into shiny, smooth pockets, but it introduces a huge amount of aliasing on minimized textures. Blending between the cases is possible with fragment programs, but the performance overhead does start piling up, and it may require stashing some information in the normal map alpha channel that varies with mip level. Doing good filtering of a specularly lit normal map texture is a fairly interesting problem, with lots of subtle issues.
Bump mapped ambient lighting will give much better looking outdoor and well-lit scenes. This only became possible with dependent texture reads, and it requires new designer and tool-chain support to implement well, so it isn't easy to test globally with the current Doom datasets, but isolated demos are promising.
The future is in floating point framebuffers. One of the most noticeable thing this will get you without fundamental algorithm changes is the ability to use a correct display gamma ramp without destroying the dark color precision. Unfortunately, using a floating point framebuffer on the current generation of cards is pretty difficult, because no blending operations are supported, and the primary thing we need to do is add light contributions together in the framebuffer. The workaround is to copy the part of the framebuffer you are going to reference to a texture, and have your fragment program explicitly add that texture, instead of having the separate blend unit do it. This is intrusive enough that I probably won't hack up the current codebase, instead playing around on a forked version.
Floating point framebuffers and complex fragment shaders will also allow much better volumetric effects, like volumetric illumination of fogged areas with shadows and additive/subtractive eddy currents.
John Carmack
-----------------------------------------
John Carmack's .plan for Feb 07, 2003
-----------------------------------------
The machinima music video that Fountainhead Entertainment (my wife's company) produced with Quake based tools is available for viewing and voting on at: http://www.mtv.com/music/viewers_pick/ ("In the waiting line")
I thought they did an excellent job of catering to the strengths of the medium, and not attempting to make a game engine compete (poorly) as a general purpose renderer. In watching the video, I did beat myself up a bit over the visible popping artifacts on the environment mapping, which are a direct result of the normal vector quantization in the md3 format. While it isn't the same issue (normals are full floating point already in Doom), it was the final factor that pushed me to do the per-pixel environment mapping for the new cards in the current engine.
The neat thing about the machinima aspect of the video is that they also have a little game you can play with the same media assets used to create the video. Not sure when it will be made available publicly.
-----------------------------------------
John Carmack's .plan for Dec 31, 2004
-----------------------------------------
Welcome
I get a pretty steady trickle of emails from people hoping for .plan file updates. There were two main factors involved in my not doing updates for a long time - a good chunk of my time and interest was sucked into Armadillo Aerospace, and the fact that the work I had been doing at Id for the last half of Doom 3 development was basically pretty damn boring.
The Armadillo work has been very rewarding from a learning-lots-of-new-stuff perspective, and I'm still committed to the vehicle development, even post X-Prize, but the work at Id is back to a high level of interest now that we are working on a new game with new technology. I keep running across topics that are interesting to talk about, and the Armadillo updates have been a pretty good way for me to organize my thoughts, so I'm going to give it a more general try here. .plan files were appropriate ten years ago, and sort of retro-cute several years ago, but I'll be sensible and use the web.
I'm not quite sure what the tone is going to be - there will probably be some general interest stuff, but a bunch of things will only be of interest to hardcore graphics geeks.
I have had some hesitation about doing this because there are a hundred times as many people interested in listening to me talk about games / graphics / computers as there are people interested in rocket fabrication, and my mailbox is already rather time consuming to get through.
If you really, really want to email me, add a "[JC]" in the subject header so the mail gets filtered to a mailbox that isn't clogged with spam. I can't respond to most of the email I get, but I do read everything that doesn't immediately scan as spam. Unfortunately, the probability of getting an answer from me doesn't have a lot of correlation with the quality of the question, because what I am doing at the instant I read it is more dominant, and there is even a negative correlation for "deep" questions that I don't want to make an off-the-cuff response to.
Quake 3 Source
I intended to release the Q3 source under the GPL by the end of 2004, but we had another large technology licensing deal go through, and it would be poor form to make the source public a few months after a company paid hundreds of thousands of dollars for full rights to it. True, being public under the GPL isn't the same as having a royalty free license without the need to disclose the source, but I'm pretty sure there would be some hard feelings.
Previous source code releases were held up until the last commercial license of the technology shipped, but with the evolving nature of game engines today, it is a lot less clear. There are still bits of early Quake code in Half Life 2, and the remaining licensees of Q3 technology intend to continue their internal developments along similar lines, so there probably won't be nearly as sharp a cutoff as before. I am still committed to making as much source public as I can, and I won't wait until the titles from the latest deal have actually shipped, but it is still going to be a little while before I feel comfortable doing the release.
Random Graphics Thoughts
Years ago, when I first heard about the inclusion of derivative instructions in fragment programs, I couldn't think of anything off hand that I wanted them for. As I start working on a new generation of rendering code, uses for them come up a lot more often than I expected.
I can't actually use them in our production code because it is an Nvidia-only feature at the moment, but it is convenient to do experimental code with the nv_fragment_program extension before figuring out various ways to build funny texture mip maps so that the built in texture filtering hardware calculates a value somewhat like the derivative I wanted.
If you are basically just looking for plane information, as you would for modifying things with texture magnification or stretching shadow buffer filter kernels, the derivatives work out pretty well. However, if you are looking at a derived value, like a normal read from a texture, the results are almost useless because of the way they are calculated. In an ideal world, all of the samples to be differenced would be calculated at once, then the derivatives calculated from there, but the hardware only calculates 2x2 blocks at a time. Each of the four pixels in the block is given the same derivative, and there is no influence from neighboring pixels. This gives derivative information that is basically half the resolution of the screen and sort of point sampled. You can often see this effect with bump mapped environment mapping into a mip-mapped cube map, where the texture LOD changes discretely along the 2x2 blocks. Explicitly coloring based on the derivatives of a normal map really shows how nasty the calculated value is.
Speaking of bump mapped environment sampling.. I spent a little while tracking down a highlight that I thought was misplaced. In retrospect it is obvious, but I never considered the artifact before: With a bump mapped surface, some of the on-screen normals will actually be facing away from the viewer. This causes minor problems with lighting, but when you are making a reflection vector from it, the vector starts reflecting into the opposite hemisphere, resulting in some sky-looking pixels near bottom edges on the model. Clamping the surface normal to not face away isn't a good solution, because you get areas that "see right through" to the environment map, because a reflection past a clamped perpendicular vector doesn't change the viewing vector. I could probably ramp things based on the geometric normal somewhat, and possibly pre-calculate some data into the normal maps, but I decided it wasn't a significant enough issue to be worth any more development effort or speed hit.
Speaking of cube maps.. The edge filtering on cube maps is showing up as an issue for some algorithms. The hardware basically picks a face, then treats it just like a 2D texture. This is fine in the middle of the texture, but at the edges (which are a larger and larger fraction as size decreases) the filter kernel just clamps instead of being able to sample the neighbors in an adjacent cube face. This is generally a non-issue for classic environment mapping, but when you start using cube map lookups with explicit LOD bias inputs (say, to simulate variable specular powers into an environment map) you can wind up with a surface covered with six constant color patches instead of the smoothly filtered coloration you want. The classic solution would be to implement border texels, but that is pretty nasty for the hardware and API, and would require either the application or the driver to actually copy the border texels from all the other faces. Last I heard, upcoming hardware was going to start actually fetching from the other side textures directly. A second-tier chip company claimed to do this correctly a while ago, but I never actually tested it.
Topics continue to chain together, I'll probably write some more next week.
-----------------------------------------
John Carmack's .plan for May 27, 2005
-----------------------------------------
Cell phone adventures
I'm not a cell phone guy. I resisted getting one at all for years, and even now I rarely carry it. To a first approximation, I don't really like talking to most people, so I don't go out of my way to enable people to call me. However, a little while ago I misplaced the old phone I usually take to Armadillo, and my wife picked up a more modern one for me. It had a nice color screen and a bunch of bad java game demos on it. The bad java games did it.
I am a big proponent of temporarily changing programming scope every once in a while to reset some assumptions and habits. After Quake 3, I spent some time writing driver code for the Utah-GLX project to give myself more empathy for the various hardware vendors and get back to some low-level register programming. This time, I decided I was going to work on a cell phone game.
I wrote a couple java programs several years ago, and I was left with a generally favorable impression of the language. I dug out my old "java in a nutshell" and started browsing around on the web for information on programming for cell phones. After working my way through the alphabet soup of J2ME, CLDC, and MIDP, I've found that writing for the platform is pretty easy.
In fact, I think it would be an interesting environment for beginning programmers to learn on. I started programming on an Apple II a long time ago, when you could just do an "hgr" and start drawing to the screen, which was rewarding. For years, I've had misgivings about people learning programming on Win32 (unix / X would be even worse), where it takes a lot of arcane crap just to get to the point of drawing something on the screen and responding to input. I assume most beginners wind up with a lot of block copied code that they don't really understand.
All the documentation and tools needed are free off the web, and there is an inherent neatness to being able to put the program on your phone and walk away from the computer. I wound up using the latest release of NetBeans with the mobility module, which works pretty well. It certainly isn't MSDev, but for a free IDE it seems very capable. On the downside, MIDP debugging sessions are very flaky, and there is something deeply wrong when text editing on a 3.6 ghz processor is anything but instantaneous.
I spent a while thinking about what would actually make a good game for the platform, which is a very different design space than PCs or consoles. The program and data sizes are tiny, under 200k for java jar files. A single texture is larger than that in our mainstream games. The data sizes to screen ratios are also far out of the range we are used to. A 128x128x16+ bit color screen can display some very nice graphics, but you could only store a half dozen uncompressed screens in your entire size budget. Contrast with PCs, which may be up to a few megabytes of display data, but the total game data may be five hundred times that.
You aren't going to be able to make an immersive experience on a 2" screen, no matter what the graphics look like. Moody and atmospheric are pretty much out. Stylish and fun is about the best you can do.
The standard cell phone style discrete button direction pad with a center action button is a good interface for one handed navigation and selection, but it sucks for games, where you really want a game boy style rocking direction pad for one thumb, and a couple separate action buttons for the other thumb. These styles of input are in conflict with each other, so it may never get any better. The majority of traditional action games just don't work well with cell phone style input.
Network packet latency is bad, and not expected to be improving in the foreseeable future, so multiplayer action games are pretty much out (but see below).
I have a small list of games that I think would work out well, but what I decided to work on is DoomRPG - sort of Bard's Tale meets Doom. Step based smooth sliding/turning tile movement and combat works out well for the phone input buttons, and exploring a 3D world through the cell phone window is pretty neat. We talked to Jamdat about the business side of things, and hired Fountainhead Entertainment to turn my proof-of-concept demo and game plans into a full-featured game.
So, for the past month or so I have been spending about a day a week on cell phone development. Somewhat to my surprise, there is very little internal conflict switching off from the high end work during the day with gigs of data and multi-hundred instruction fragment shaders down to texture mapping in java at night with one table lookup per pixel and 100k of graphics. It's all just programming and design work.
It turns out that I'm a lot less fond of Java for resource-constrained work. I remember all the little gripes I had with the Java language, like no unsigned bytes, and the consequences of strong typing, like no memset, and the inability to read resources into anything but a char array, but the frustrating issues are details down close to the hardware.
The biggest problem is that Java is really slow. On a pure cpu / memory / display / communications level, most modern cell phones should be considerably better gaming platforms than a Game Boy Advanced. With Java, on most phones you are left with about the CPU power of an original 4.77 mhz IBM PC, and lousy control over everything.
I spent a fair amount of time looking at java byte code disassembly while optimizing my little rendering engine. This is interesting fun like any other optimization problem, but it alternates with a bleak knowledge that even the most inspired java code is going to be a fraction the performance of pedestrian native C code.
Even compiled to completely native code, Java semantic requirements like range checking on every array access hobble it. One of the phones (Motorola i730) has an option that does some load time compiling to improve performance, which does help a lot, but you have no idea what it is doing, and innocuous code changes can cause the compilable heuristic to fail.
Write-once-run-anywhere. Ha. Hahahahaha. We are only testing on four platforms right now, and not a single pair has the exact same quirks. All the commercial games are tweaked and compiled individually for each (often 100+) platform. Portability is not a justification for the awful performance.
Security on a cell phone is justification for doing something, but an interpreter isn't a requirement - memory management units can do just as well. I suspect this did have something to do with Java's adoption early on. A simple embedded processor with no MMU could run arbitrary programs securely with java, which might make it the only practical option. However, once you start using blazingly fast processors to improve the awful performance, a MMU with a classic OS model looks a whole lot better.
Even saddled with very low computing performance, tighter implementation of the platform interface could help out a lot. I'm not seeing very conscientious work on the platforms so far. For instance, there is just no excuse for having 10+ millisecond granularity in timing. Given that the java paradigm is sort of thread-happy anyway, having a real scheduler that Does The Right Thing with priorities and hardware interfacing would be an obvious thing. Pressing a key should generate a hardware interrupt, which should immediately activate the key listening thread, which should be able to immediately kill an in-process rendering and restart another one if desired. The attitude seems to be 15 msec here, 20 there, stick it on a queue, finish up a timeslice, who cares, right?
I suspect I will enjoy working with BREW, the competing standard for cell phone games. It lets you use raw C/C++ code, or even, I suppose, assembly language, which completely changes the design options. Unfortunately, they only have a quarter the market share that the J2ME phones have. Also, the relatively open java platform development strategy is what got me into this in the first place - one night I just tried writing a program for my cell phone, which isn't possible for the more proprietary BREW platform.
I have a serious suggestion for the handset designers to go with my idle bitching. I have been told that fixing data packet latency is apparently not in the cards, and it isn't even expected to improve much with the change to 3G infrastructure. Packet data communication seems more modern, and has the luster of the web, but it is worth realizing that for network games and many other flashy Internet technologies like streaming audio and video, we use packets to rather inefficiently simulate a switched circuit.
Cell phones already have a very low latency digital data path - the circuit switched channel used for voice. Some phones have included cellular modems that use either the CSD standard (circuit switched data) at 9.8Kbits or 14.4Kbits or the HSCSD standard (high speed circuit switched data) at 38.4Kbits or 57.6Kbits. Even the 9.8Kbit speed would be great for networked games. A wide variety of two player peer-to-peer games and multiplayer packet server based games could be implemented over this with excellent performance. Gamers generally have poor memories of playing over even the highest speed analog modems, but most of the problems are due to having far too many buffers and abstractions between the data producers/consumers and the actual wire interface. If you wrote eight bytes to the device and it went in the next damned frame (instead of the OS buffer, which feeds into a serial FIFO, which goes into another serial FIFO, which goes into a data compressor, which goes into an error corrector, and probably a few other things before getting into a wire frame), life would be quite good. If you had a real time scheduler, a single frame buffer would be sufficient, but since that isn't likely to happen, having an OS buffer with accurate queries of the FIFO positions is probably best. The worst gaming experiences with modems weren't due to bandwidth or latency, but to buffer pileup.
-----------------------------------------
John Carmack's .plan for May 02, 2006
-----------------------------------------
Orcs & Elves
I'm not managing to make regular updates here, but I'll keep this around just in case. I have a bunch of things that I want to talk about -- some thoughts on programming style and reliability, OpenGL, Xbox 360, etc, but we have a timely topic with the release of our second mobile game, Orcs & Elves, that has spurred me into making this update.
DoomRPG, our (Id Software's and Fountainhead Entertainment's) first mobile title, has been very successful, both in sales and in awards. I predict that the interpolated turn based style of 3D gaming will be widely adopted on the mobile platform, because it plays very naturally on a conventional cell phone. Gaming will be a lot better when there is a mass market of phones that can be played more like a gamepad, but you need to make do with what you actually have.
One of the interesting things about mobile games is that the sales curve is not at all like the drastically front loaded curve of a PC or console game. DoomRPG is selling better now than when it was initially released, and the numbers are promising for supporting additional development work. However, unless I am pleasantly surprised, the hardware capabilities are going to advance much faster than the market in the next couple years, leading to an unusual situation where you can only afford to develop fairly crude games on incredibly powerful hardware. Perhaps "elegantly simple" would be the better way of looking at it, but it will still wind up being like developing an Xbox title for $500,000. That will wind up being great for many small game companies that just want to explore an idea, but having resource far in excess of your demands does minimize the value of being a hot shot programmer. :-)
To some degree this is already the case on high end BREW phones today. I have a pretty clear idea what a maxed out software renderer would look like for that class of phones, and it wouldn't be the PlayStation-esq 3D graphics that seems to be the standard direction. When I was doing the graphics engine upgrades for BREW, I started along those lines, but after putting in a couple days at it I realized that I just couldn't afford to spend the time to finish the work. "A clear vision" doesn't mean I can necessarily implement it in a very small integral number of days. I wound up going with a less efficient and less flexible approach that was simple and robust enough to not likely need any more support from me after I handed it over (it didn't).
During the development of DoomRPG, I had commented that it seemed obvious that it should be followed up with a "traditional, Orcs&Elves sort of fantasy game". A couple people independently commented that "Orcs&Elves" wasn't a bad name for a game so since we didn't run into any obstacles, Orcs& Elves it was. Naming new projects is a lot harder than most people think, because of trademark issues.
In hindsight, we made a strategic mistake at the start of O&E development. We were fresh off the high end BREW version of DoomRPG, and we all liked developing on BREW a lot better than Java. It isn't that BREW is inherently brilliant, it just avoids the deep sucking nature of java for resource constrained platforms (however, note the above about many mobile games not being resource constrained in the future), and allows you to work inside visual studio. O&E development was started high-end first with the low-end versions done afterwards. I should have known better (Anna was certainly suspicious), because it is always easier to add flashy features without introducing any negatives than it is to chop things out without damaging the core value of a game. The high end version is really wonderful, with all the graphics, sound, and gameplay we aimed for, but when we went to do the low end versions, we found that even after cutting the media as we planned, we were still a long way over the 280k java application limit. Rather than just butchering it, we went for pain, suffering, and schedule slippage, eventually delivering a game that still maintained high quality after the de-scoping (the low end platforms still represent the majority of the market). It would have been much easier to go the other way, but the high end phone users will be happy with our mistake.
DoomRPG had three base platforms that were customized for different phones -- Java, low end BREW, and high end BREW. O&E added a high end java version that kept most of the quality of the high end BREW version on phones fast enough to support it from carriers willing to allow the larger download. The download size limits are probably the most significant restriction for gaming on the high end phones. I don't really understand why the carriers encourage streaming video traffic, but balk at a couple megs of game media.
I am really looking forward to the response to Orcs&Elves, because I think it is one of the best product evolutions I have been involved in. The core game play mechanics that were laid out in DoomRPG have proven strong and versatile (again, I bet we have a stable genre here), but now we have a big bag of tricks and a year of polishing the experience behind us, along with a world of some depth. I found it a very good indicator that play testers almost always lost track of time while playing.
This project was doubly nostalgic for me -- the technology was over a decade old for me, but the content took me back twenty years. All the computer games I wrote in high school were adventure games, and my first two commercial sales were Ultima style games for the Apple II, but Id Software never got around to doing one. Old timers may recall that we were going to do a fantasy game called "The Fight For Justice" (starring a hero called Quake...) after Commander Keen, but Wolfenstein 3D and the birth of the FPS sort of got in the way. :-)
-----------------------------------------
John Carmack's .plan for Nov 02, 2007
-----------------------------------------
Technology and Games
Source: http://blogs.ign.com/OrcsandElves/2007/11/02/70591/
Most of my time lately is spent working on Rage, id Software's Id Tech 5 based game that runs on PCs, Macs, 360s, and PS3s. A modern high-end game is a really breathtaking consumer of computation and storage resources for anyone that has been around computers for any length of time. Our target platforms have at least 512 mb of ram, almost 20 gb of media storage, and many tens of gflops of computation, but the development environment involves an even more massive deployment, with a terabyte of raw data being generated before the final culling and compression is done. It is easy to be a little nonchalant about the continuous pace of improvement with computing, but I still take the time to feel a sense of awe about it all.
I started programming on a Commodore VIC-20 with 4k of ram and a tape drive, and I remember writing absurdly long lines of basic code to save the couple bytes that a new line number would consume. As the years went by, and my projects moved from the Apple II to the PC and the early consoles, I continued to gain new insights and perspective on different problems, and I often thought that it would be fun to go back to one of the early small systems. Now that I "knew what I was doing", I could do a lot more within the tight constraints than I was ever able to before. I actually carried a bunch of old systems around with me from house to house for many years before I reached the conclusion that I never was going to spend any real time on them again, and finally tossed them out.
As technology continued to rapidly advance, I saw a lot of good programmers sort of peel off from high end game development, and migrate to other platforms where their existing skill sets were still exactly what was needed. There was a large contingent of hard core assembly language programmers that never wanted to "get soft" with the move to C for more complex game development, and many of them moved from PCs to platforms like the super Nintendo, and eventually into embedded systems or device firmware. There was another contingent that never wanted to move to windows, and so on.
There is an appeal to working with tighter constraints. The more limited the platform, the closer you can feel you are getting to an "optimal" solution. On a modern big system, there are dozens of ways to accomplish any given task, and it just isn't possible to evaluate all the tradeoffs between the implementations of hundreds of different tasks. On a little system, you have to constrain your design to have a much smaller total number of tasks, and the available options are a lot more reduced. Seeing if something is The Right Thing is a lot easier.
I probably had my personal "moment of truth" around the beginning of Doom 3's development, when it became clear that it is no longer possible to deeply understand every single part of a modern application. There is just too much. Nevertheless, I found that I could still enjoy my work when confined to a subset of the entire project, and I have thus remained committed to the high end. However, the appeal of smaller systems still lingers.
A couple years ago, almost on a whim, I got involved in developing games for mobile phones. The primary instigator was when I ran a bunch of games on a new phone and was left frankly appalled at how poor they were. I was thinking to myself "I wrote better games than this each month before we founded Id". I downloaded the SDK for the phone, and started tinkering around a bit. A proof of concept demo and a plan for a game play style specifically tailored for cell phones followed, and we wound up with DoomRPG, and later Orcs & Elves which turned out to be big hits.
In an ideal world, where I could either stop time or clone myself, I would act as a lead programmer for some smaller projects. In the real world, I can't justify spending much time away from the high-end work, so the low-end work gets done in a few short bursts of engine creation and foundation laying, which is then handed over to the Fountainhead team to actually build a great game. After that, Anna mostly uses me as a threat -- if her programmers tell her that something she really wants in a game can't be done, she threatens to call me up and have me tell them how straightforward the problem really is, which usually fuels them to figure out how to do it on their own.
Mobile development was fun in a resource constrained design sort of way, but the programming didn't have the "tight" feel that early gaming had, due to the huge variability in the platforms. When I was messing around on my own phone, I spent some time doing java bytecode disassembly and timing, but it was fairly pointless in light of the two hundred or so different phones the game would wind up running on.
Enter the Nintendo DS.
We had initially looked at possibly moving one of the cell phone titles over to the GBA, but it didn't look like the market would support it, and technically it would have only turned out somewhere in between the low end and high end cell phone versions. With the success of DS, and a suspicion that the players might be a little more open to new third party titles, EA decided that they would support developing a really enhanced version of Orcs&Elves for the DS. This is going a bit out on a limb -- most successful Game Boy / DS titles have been either first-party Nintendo titles, or titles with a strong movie / toy / history tie in. While Orcs & Elves is doing well on mobile, it is still very far from a recognized brand.
The resource limits on the DS make it almost perfect for a small team development. The hardware is fun to work with, you can directly poke at all the registers, the tool chain works well, and the built in limitations of cartridge memory keeps the design decisions fairly straightforward. Going one step farther up to the PSP with a UMD brings you into the realm of large media sizes that can rapidly consume multi-million dollar development budgets.
Once the decision was made to go for it, it was my job to figure out what we could reasonably hope to accomplish on the platform, andbring up a first cut at the 3D rendering engine.
Up next: all the technical details
-----------------------------------------
John Carmack's .plan for Nov 08, 2007
-----------------------------------------
DS Technology
Source: http://blogs.ign.com/OrcsandElves/2007/11/08/71156/
The actual implementation decisions for Orcs&Elves DS were driven by the hardware, the development timeline, and the budget. With a five person team, we had six months to bring it all together on a new platform. I wrote the code for the hardware accelerated 3D renderer and for the remainder of the project, I was technical advisor.
The basic compute power of a 32 bit, 66 mhz arm processor and four megs of ram is a pleasant size to work with, basically about what we had on the PC back when the original Doom was written. You are intrinsically limited to a design that is compact enough that you can wrap your head around every aspect of it at once, but you don't wind up mired in crazy size decisions like trading a couple functions of code for an extra graphical icon.
Going back to fixed point math is always a chore, and while the DS has hardware acceleration for fixed point math, it doesn't automatically integrate with C/C++ code. The compiler / linker / debugger tool chain worked just fine, and I never felt that I was fighting the development environment like it used to be with the really early consoles. The DS SDK takes my preferred approach of both documenting the hardware fully and providing a helper library with full source code that you can pull apart as necessary.
The baseline spec we started with was a 16 meg cart for the game. I was sure we could get all the features we were discussing to fit in there, but in hindsight, we really should have pushed for a 32 meg cart, because it would have allowed us to add a lot more high quality 2D art to the game, and include some pre-rendered cinematics to help set the mood and tell the story. Anna had pushed for this from the beginning, but I was worried that we wouldn't have enough time to create the additional media, and I didn't want to eat the extra manufacturing costs on a speculative game release of an unknown IP.
At first glance, a 16 meg cart is over eight times as large as our high end cell phone distribution, but that turns out to be misleading. Everything is highly compressed on the mobile versions, but because of the need to directly DMA many assets in a usable form on the DS, and sometimes due to the tighter ram limits, a lot of the media takes up more space on the DS. The game is still a lot bigger, but not 8x.
Interfacing with the DS 3D graphics processor was my major contribution to the project. The DS is an unusual graphics machine, with no direct analog before it, but the individual pieces were close enough to things I had experience with that I was able to be effective pretty quickly. While there are a few things I wish would have been done differently, I still found it a lot of fun to work with.
The geometry engine is nice and clean, implementing a good subset of the basic OpenGL pipe in fixed point. It is also quite fast relative to the rest of the system. I had originally laid out the code to double buffer all the command traffic so that the geometry engine could completely overlap with the CPU, but it turned out that we would run into the polygon limits on the rasterizer before the geometry engine worked up much of a sweat, so I just saved memory and let the geometry engine run with a single command buffer. The game logic tended to take more time to process than the geometry engine, so it was almost never a gating factor. A title that heavily used vertex lighting and matrix stack operations might be able to load up the geometry pipeline a bit, but with just texture and color at the verts, it has margin.
Unlike classic OpenGL, you can change the matrix in the middle of specifying a primitive, allowing single bone skinned model rendering to be performed with a conventional vertex pipeline. This was a novel approach that I hadn't seen anywhere else. It was a little odd to see an explicitly separate model and projection matrix in hardware, since they are usually pre-multiplied into a single MVP matrix, but it allows the slow main cpu to avoid having to do some matrix math.
The question of 3D models versus sprites was one of the most significant decisions for Orcs&Elves. There are several situations in the game where you may be facing six or eight monsters, and I was concerned about how low poly the enemies would have to be to avoid problems with the rasterizer, especially when you consider that monsters can chase you into scenes that may tax the rasterizer all by themselves. Coupled with the fact that developing a new skeletal animation system, building all new models, and animating them would almost certainly have busted our development timeline, in the given circumstances, we decided sprites would do a better job.
The rasterization side of things on the DS is... quirky. Almost all 3D rendering systems today use a frame buffer and a depth buffer that is stored in dedicated graphics memory and incrementally updated by the graphics processor. The DS essentially renders as it is being displayed, with only a small set of lines acting as a buffer. This saves memory, and can give speed and power savings, but it has some significant tradeoffs.
Failure modes are bad -- If you overload the polygon list, the remaining polygons just don't show up. This can be mediated by drawing the nearby and important things first, so if anything disappears it is hopefully in the distance where it isn't very noticeable. If you overload the fill rate, horizontal bars appear across the screen. In a perspective view, the greatest complexity and density tend to be in the middle of the screen, so if your scene gets too bad, a jittering color bar tends to appear in the center.
In a conventional rendering engine, overloading complexity just tends to make the game go slower. On the DS, overloading looks broken. This means that you need to leave yourself a lot more margin, and therefore use the hardware less aggressively. The plus side is that since the hardware is re-drawing the screen at 60hz no matter what you do, you are strongly encouraged to make sure the rest of your game also stays at 60hz.
The lack of texture filtering on the DS is the most obvious difference with other current platforms, and it does strongly encourages a cartoony art style for games. The art style for O&E isn't really ideal, and perhaps we should have stylized things a bit more.
You don't have a lot of texture memory available on the DS, less even than an original Playstation. In O&E, the environment graphics (walls and floors) are allocated statically at map load time, and must be balanced by the level artist. The character sprites are dynamically managed, with high resolution artwork for the monster directly in front of the player being swapped in every frame. The field of view and sprite positioning are carefully managed, along with a "pushback factor", to ensure that the sprites are at 1:1 scale when they are one tile away, and 2:1 scale when they are right in front of the player. Without bilinear filtering on the texturing, non-integral scales wind up looking very ugly. This is one of the advantages of the tile based play -- if it was completely free form, the monsters would look a lot uglier. There isn't any intermediate step to be taken to improve the monster rendering without using a full 4x the memory to render an adjacent monster at a 1:1 scale. Even if we had more memory, I probably would have spent it on more animations instead of higher resolution.
The most disappointing mistake in the DS hardware is the lack of even the basic original blending modes from OpenGL. This was a mistake that was common in the very first generation of PC 3D accelerators, where so many companies just assumed that "blending" meant "alpha blending", and they didn't include support for add, modulate, and the other standard blending modes. No relevant company had made that mistake in a decade, and Nintendo's consoles have always done it right since the U64, so it was a surprise to see it in the DS. Additive blending in particular is crucial to most "3D flash" sorts of rendering, and various non-additive modulation modes are used for light mapping and other core rendering features.
I only got to spend four days actually writing all the 3D code for Orcs&Elves, so there are lots of potential directions that I am interested in exploring in the future. We plan to have two more DS projects in development next year, which I hope will let me try out a skeletal animation system, experiment with the networking hardware, and implement a more flexible high level culling algorithm than what I used in O&E.
John Carmack
-----------------------------------------
John Carmack's .plan for Mar 26, 2009
-----------------------------------------
iPhone development: Wolfenstein 3D Classic
Source: http://www.idsoftware.com/wolfenstein3dclassic/wolfdevelopment.htm
I had been frustrated for over a year with the fact that we didn't have any iPhone development projects going internally at Id. I love my iPhone, and I think the App Store is an extremely important model for the software business. Unfortunately, things have conspired against us being out early on the platform.
Robert Duffy and I spent a week early on starting to bring up the Orcs & Elves DS codebase on the iPhone, which would have been a nice project for a launch title, but it wasn't going to be a slam dunk. The iPhone graphics hardware is a more capable superset of the DS hardware (the driver overhead is far, far worse, though), but the codebase was fairly DS specific, with lots of Nintendo API calls all over the place. I got the basics drawing by converting things to OpenGL ES, but I was still on the fence as to whether the best approach to get all the picky little special effects working would be a complete GL conversion, or a DS graphics library emulation layer. Coupled with the fact that the entire user interface would need to be re-thought and re-tested, it was clear that the project would take several months of development time, and need artists and designers as well as coding work. I made the pitch that this would still be a good plan, but the idMobile team was already committed to the Wolfenstein RPG project for conventional Java and BREW mobile phones, and Anna didn't want to slip a scheduled milestone on the established, successful development directions there for a speculative iPhone project.
After thinking about the platform's capabilities a bit more, I had a plan for an aggressive, iPhone specific project that we actually started putting some internal resources on, but the programmer tasked with it didn't work out and was let go. In an odd coincidence, an outside development team came to us with a proposal for a similar project on the Wii, and we decided to have them work on the iPhone project with us instead. We should be announcing this project soon, and it is cool. It is also late, but that's software development...
Late last year, the mobile team had finished up all the planned versions of Wolfenstein RPG, but EA had suggested that in addition to the hundreds of customized versions they normally produce for all the various mobile phones, they were interested in having another team do a significant media quality improvement on it for the iPhone. While Wolf RPG is a very finely crafted product for traditional cell phones, it wasn't designed for the iPhone's interface or capabilities, so it wouldn't be an ideal project, but it should still be worth doing. When we got the first build to test, I was pleased with how the high res artwork looked, but I was appalled at how slow it ran. It felt like one of the mid range java versions, not better than the high end BREW as I expected. I started to get a sinking feeling. I searched around in the level for a view that would confirm my suspicion, and when I found a clear enough view of some angled geometry I saw the tell-tale mid-polygon affine swim in the texture as I rotated. They were using the software rasterizer on the iPhone. I patted myself on the back a bit for the fact that the combination of my updated mobile renderer, the intelligent level design / restricted movement, and the hi-res artwork made the software renderer almost visually indistinguishable from a hardware renderer, but I was very unhappy about the implementation.
I told EA that we were NOT going to ship that as the first Id Software product on the iPhone. Using the iPhone's hardware 3D acceleration was a requirement, and it should be easy -- when I did the second generation mobile renderer (written originally in java) it was layered on top of a class I named TinyGL that did the transform / clip / rasterize operations fairly close to OpenGL semantics, but in fixed point and with both horizontal and vertical rasterization options for perspective correction. The developers came back and said it would take two months and exceed their budget.
Rather than having a big confrontation over the issue, I told them to just send the project to me and I would do it myself. Cass Everitt had been doing some personal work on the iPhone, so he helped me get everything set up for local iPhone development here, which is a lot more tortuous than you would expect from an Apple product. As usual, my off the cuff estimate of "Two days!" was optimistic, but I did get it done in four, and the game is definitely more pleasant at 8x the frame rate.
And I had fun doing it.
Since we now were doing something resembling "real work" on the iPhone at the office, we kept it going at a low priority. One of the projects Cass was tinkering around with at home was a port of Quake 3, and we talked about different interface strategies every now and then.
Unfortunately, when we sat down to try a few things out, we found that Q3 wasn't really running fast enough to make good judgments on iPhone control systems. The hardware should be capable enough, but it will take some architectural changes to the rendering code to get the most out of it.
I was just starting to set up a framework to significantly revise Q3 when I considered the possibility of just going to an earlier codebase to experiment with initially. If we wanted to factor performance out of the equation, we could go all the way back to Wolfenstein 3D, the grandfather of FPS games. It had the basic run and gun play that has been built on for fifteen years, but it originally ran on 286 computers, so it should be pretty trivial to hold a good framerate on the iPhone.
Wolfenstein was originally written in Borland C and TASM for DOS, but I had open sourced the code long ago, and there were several projects that had updated the original code to work on OpenGL and modern operating systems. After a little looking around, I found Wolf3D Redux at http://wolf3dredux.sourceforge.net/. One of the development comments about "removal of the gangrenous 16 bit code" made me smile.
It was nice and simple to download, extract data from a commercial copy of Wolfenstein, and start playing on a PC at high resolution. Things weren't as smooth as they should be at first, but two little changes made a huge difference -- going at VBL synced update rates with one tic per cycle instead of counting milliseconds to match 70 hz game tics, and fixing a bug with premature integralization in the angle update code that caused mouse movement to be notchier than it should be. The game was still fun to play after all these years, and I began to think that it might be worthwhile to actually make a product out of Wolfenstein on the iPhone, rather than just using it as a testbed, assuming the controls worked out as fun to play. The simple episodic nature of the game would make it easy to split up into a $0.99 version with just the first episode, a more expensive version with all sixty levels, and we could release Spear of Destiny if there was additional demand. I was getting a little ahead of myself without a fun-to-play demonstration of feasibility on the iPhone, but the idea of moving the entire line of classic Id titles over -- Wolf, Doom, Quake, Quake 2, and Quake Arena, was starting to sound like a real good idea.
I sent an email to the Wolf 3D Redux project maintainer to see if he might be interested in working on an iPhone project with us, but it had been over a year since the last update, and he must have moved on to other things. I thought about it a bit, and decided that I would go ahead and do the project myself. The "big projects" at Id are always top priority, but the systems programming work in Rage is largely completed, and the team hasn't been gated on me for anything in a while. There is going to be memory and framerate optimization work going on until it ships, but I decided that I could spend a couple weeks away from Rage to work on the iPhone exclusively. Cass continued to help with iPhone system issues, I drafted Eric Will to create the few new art assets, and Christian Antkow did the audio work, but this was the first time I had taken full responsibility for an entire product in a very long time.
@Design notes@
The big question was how "classic" should we leave the game? I have bought various incarnations of Super Mario Bros on at least four Nintendo platforms, so I think there is something to be said for the classics, but there were so many options for improvement. The walls and sprites in the game were originally all 64 x 64 x 8 bit color, and the sound effects were either 8khz / 8 bit mono or (sometimes truly awful) FM synth sounds. Changing these would be trivial from a coding standpoint. In the end, I decided to leave the game media pretty much unchanged, but tweak the game play a little bit, and build a new user framework around the core play experience. This decision was made a lot easier by the fact that we were right around the 10 meg over-the-air app download limit with the converted media. This would probably be the only Id project to ever be within hailing distance of that mark, so we should try to fit it in.
The original in-game status bar display had to go, because the user's thumbs were expected to cover much of that area. We could have gone with just floating stats, but I thought that BJ's face added a lot of personality to the game, so I wanted to leave that in the middle of the screen. Unfortunately, the way the weapon graphics were drawn, especially the knife, caused issues if they were just drawn above the existing face graphics. I had a wider background created for the face, and used the extra space for directional damage indicators, which was a nice improvement in the gameplay. It was a tough decision to stop there on damage feedback, because a lot of little things with view roll kicks, shaped screen blends, and even double vision or blurring effects, are all pretty easy to add and quite effective, but getting farther away from "classic".
I started out with an explicit "open door" button like the original game, but I quickly decided to just make that automatic. Wolf and Doom had explicit "use" buttons, but we did away with them on Quake with contact or proximity activation on everything. Modern games have generally brought explicit activation back by situationally overriding attack, but hunting for push walls in Wolf by shooting every tile wouldn't work out. There were some combat tactics involving explicitly shutting doors that are gone with automatic-use, and some secret push walls are trivially found when you pick up an item in front of them now, but this was definitely the right decision.
You could switch weapons in Wolf, but almost nobody actually did, except for occasionally conserving ammo with the chain gun, or challenges like "beat the game with only the knife". That functionality didn't justify the interface clutter.
The concept of "lives" was still in wolf, with 1-ups and extras at certain scores. We ditched that in Doom, which was actually sort of innovative at the time, since action games on computers and consoles were still very much take-the-quarter arcade oriented. I miss the concept of "score" in a lot of games today, but I think the finite and granular nature of the enemies, tasks, and items in Wolf is better suited to end-of-level stats, so I removed both lives and score, but added persistent awards for par time, 100% kills, 100% secrets, and 100% treasures. The award alone wasn't enough incentive to make treasures relevant, so I turned them into uncapped +1 health crumbs, which makes you always happy to find them.
I increased the pickup radius for items, which avoided the mild frustration of having to sometimes make a couple passes at an item when you are cleaning up a room full of stuff.
I doubled the starting ammo on a fresh level start. If a player just got killed, it isn't good to frustrate them even more with a severe ammo conservation constraint. There was some debate about the right way to handle death: respawn with the level as is (good in that you can keep making progress if you just get one more shot off each time, bad in that weapon pickups are no longer available), respawn just as you entered the level (good -- keep your machinegun / chaingun, bad -- you might have 1 health), or, what I chose, restart the map with basic stats just as if you had started the map from the menu.
There are 60 levels in the original Wolf dataset, and I wanted people to have the freedom to easily jump around between different levels and skills, so there is no enforcement of starting at the beginning. The challenge is to /complete /a level, not /get to/ a level. It is fun to start filling in the grid of level completions and awards, and it often feels better to try a different level after a death. The only exception to the start-anywhere option is that you must find the entrance to the secret levels before you can start a new game there.
In watching the early testers, the biggest issue I saw was people sliding off doors before they opened, and having to maneuver back around to go through. In Wolf, as far as collision detection was concerned, everything was just a 64x64 tile map that was either solid or passable.
Doors changed the tile state when they completed opening or began closing. There was discussion about magnetizing the view angle towards doors, or somehow beveling the areas around the doors, but it turned out to be pretty easy to make the door tiles only have a solid central core against the player, so players would slide into the "notch" with the door until it opened. This made a huge improvement in playability.
There is definitely something to be said for a game that loads in a few seconds, with automatic save of your position when you exit. I did a lot of testing by playing the game, exiting to take notes in the iPhone notepad, then restarting Wolf to resume playing. Not having to skip through animated logos at the start is nice. We got this pretty much by accident with the very small and simple nature of Wolf, but I think it is worth specifically optimizing for in future titles.
The original point of this project was to investigate FPS control schemes for the iPhone, and a lot of testing was done with different schemes and parameters. I was sort of hoping that there would be one "obviously correct" way to control it, but it doesn't turn out to be the case.
For a casual first time player, it is clearly best to have a single forward / back / turn control stick and a fire button.
Tilt control is confusing for first exposure to the game, but I think it does add to the fun factor when you use it. I like the tilt-to-move option, but people that play a lot of driving games on the iPhone seem to like tilt-to-turn, where you are sort of driving BJ through the levels. Tilt needs a decent deadband, and a little bit of filtering is good. I was surprised that the precision on the accelerometer was only a couple degrees, which makes it poorly suited for any direct mapped usage, but it works well enough as a relative speed control.
Serious console gamers tend to take to the "dual stick" control modes easily for movement, but the placement of the fire button is problematic. Using an index finger to fire is effective but uncomfortable. I see many players just move the thumb to fire, using strafe movement for fine tuning aim. It is almost tempting to try to hijack the side volume switch for fire, but the ergonomics aren't quite right, and it would be very un-Apple-like, and wouldn't be available on the iPod touch (plus I couldn't figure out how...).
We tried a tilt-forward to fire to allow you to keep your thumbs on the dual control sticks, but it didn't work out very well. Forward / back tilt has the inherent variable holding angle problem for anything, and a binary transition point is hard for people to hold without continuous feedback. Better visual feedback on the current angle and trip point would help, but we didn't pursue it much. For a game with just, say, a rocket launcher, shake/shove-to-fire might be interesting, but it isn't any good for wolf.
It was critical for the control sticks to be analog, since digital direction pads have proven quite ineffective on touch screens due to progressive lack of registration during play. With an analog stick, the player has continuous visual feedback of the stick position in most cases, so they can self correct. Tuning the deadband and slide off behavior are important.
Level design criteria has advanced a lot since Wolfenstein, but I wasn't going to open up the option of us modifying the levels, even though the start of the first level is painfully bad for a first time player, with the tiny, symmetric rooms for them to get their nose mashed into walls and turned around in. The idea is that you started the game in a prison cell after bashing your guard over the head, but even with the exact same game tools, we would lead the player through the experience much better now. Some of the levels are still great fun to play, and it is interesting to read Tom Hall and John Romero's designer notes in the old hint manuals, but the truth is that some levels were scrubbed out in only a couple hours, unlike the long process of testing and adjustment that goes on today.
It was only after I thought I was basically done with the game that Tim Willits pointed out the elephant in the gameplay room -- for 95% of players, wandering around lost in a maze isn't very much fun.
Implementing an automap was pretty straightforward, and it probably added more to the enjoyment of the game than anything else. Before adding this, I thought that only a truly negligible amount of people would actually finish all 60 levels, but now I think there might be enough people that get through them to justify bringing the Spear of Destiny levels over later.
When I was first thinking about the project I sort of assumed that we wouldn't bother with music, but Wolf3D Redux already had code that converted the old id music format into ogg, so we would up with support at the beginning, and it turned out pretty good. We wound up ripping the red book audio tracks from one of the later commercial Wolf releases and encoding at a different bitrate, but I probably wouldn't have bothered if not for the initial support. It would have been nice to re-record the music with a high quality MIDI synth, but we didn't have the original MIDI source, and Christian said that the conversion back from the id music format to midi was a little spotty, and would take a fair amount of work to get right. I emailed Bobby Prince, the original composer, to see if he had any high quality versions still around, but he didn't get back with me.
The game is definitely simplistic by modern standards, but it still has its moments. Getting the drop on a brown shirt just as he is pulling his pistol from the holster. Making an SS do the "twitchy dance" with your machine gun. Rounding a corner and unloading your weapon on ... a potted plant. Simplistic plays well on the iPhone.
@Programming notes@
Cass and I got the game running on the iPhone very quickly, but I was a little disappointed that various issues around the graphics driver, the input processing, and the process scheduling meant that doing a locked-at-60-hz game on the iPhone wasn't really possible. I hope to take these up with Apple at some point in the future, but it meant that Wolf would be a roughly two tick game. It is only "roughly" because there is no swapinterval support, and the timer scheduling has a lot of variability in it. It doesn't seem to matter all that much, the play is still smooth and fun, but I would have liked to at least contrast it with the perfect limit case.
It turns out that there were a couple issues that required work even at 30hz. For a game like Wolf, any PC that is in use today is essentially infinitely fast, and the Wolf3D Redux code did some things that were convenient but wasteful. That is often exactly the right thing to do, but the iPhone isn't quite as infinitely fast as a desktop PC.
Wolfenstein (and Doom) originally drew the characters as sparse stretched columns of solid pixels (vertical instead of horizontal for efficiency in interleaved planar mode-X VGA), but OpenGL versions need to generate a square texture with transparent pixels. Typically this is then drawn by either alpha blending or alpha testing a big quad that is mostly empty space. You could play through several early levels of Wolf without this being a problem, but in later levels there are often large fields of dozens of items that stack up to enough overdraw to max out the GPU and drop the framerate to 20 fps. The solution is to bound the solid pixels in the texture and only draw that restricted area, which solves the problem with most items, but Wolf has a few different heavily used ceiling lamp textures that have a small lamp at the top and a thin but full width shadow at the bottom. A single bounds doesn't exclude many texels, so I wound up including two bounds, which made them render many times faster.
The other problem was CPU related. Wolf3d Redux used the original ray casting scheme to find out which walls were visible, then called a routine to draw each wall tile with OpenGL calls. The code looked something like this:
[code]DrawWall( int wallNum ) {
char name[128];
texture_t *tex;
sprintf( name, "walls/%d.tga", wallNum );
tex = FindTexture( name );
...
}
texture_t FindTexture( const char *name ) {
int i;
for ( i = 0 ; i < numTextures ; i++ ) {
if ( !strcmp( name, texture[name]->name ) ) {
return texture[name];
}
}
...
} [/code]
I winced when I saw that at the top of the instruments profile, but again, you could play all the early levels that only had twenty or thirty visible tiles at a time without it actually being a problem. However, some later levels with huge open areas could have over a hundred visible tiles, and that led to 20hz again. The solution was a trivial change to something resembling:
[code]DrawWall( int wallNum ) {
texture_t *tex = wallTextures[wallNum];
...
} [/code]
Wolf3D Redux included a utility that extracted the variously packed media from the original games and turned them into cleaner files with modern formats. Unfortunately, an attempt at increasing the quality of the original art assets by using hq2x graphics scaling to turn the 64x64 art into better filtered 128x128 arts was causing lots of sprites to have fringes around them due to incorrect handling of alpha borders. It wasn't possible to fix it up at load time, so I had to do the proper outline-with-color-but-0-alpha operations in a modified version of the extractor. I also decided to do all the format conversion and mip generation there, so there was no significant CPU time spent during texture loading, helping to keep the load time down. I experimented with the PVRTC formats, but while it would have been ok for the walls, unlike with DXT you can't get a lossless alpha mask out of it, so it wouldn't have worked for the sprites. Besides, you really don't want to mess with the carefully chosen pixels in a 64x64 block very much when you scale it larger than the screen on occasion.
I also had to make one last minute hack change to the original media -- the Red Cross organization had asserted their trademark rights over red crosses (sigh) some time after we released the original Wolfenstein 3D game, and all new game releases must not use red crosses on white backgrounds as health symbols. One single, solitary sprite graphic got modified for this release.
User interface code was the first thing I started making other programmers do at Id when I no longer had to write every line of code in a project, because I usually find it tedious and unrewarding. This was such a small project that I went ahead and did it myself, and I learned an interesting little thing. Traditionally, UI code has separate drawing and input processing code, but on a touchscreen device, it often works well to do a combined "immediate mode interface", with code like this:
[code]if ( DrawPicWithTouch( x, y, w, h, name ) ) {
menuState = newState;
} [/code]
Doing that for the floating user gameplay input controls would introduce a frame of response latency, but for menus and such, it works very well.
One of the worst moments during the development was when I was getting ready to hook up the automatic savegame on app exit. There wasn't any savegame code. I went back and grabbed the original 16 bit dos code for load / save game, but when I compiled I found out that the Wolf3d Redux codebase had changed a lot more than just the near / far pointer issues, asm code, and comment blocks. The changes were sensible things, like grouping more variables into structures and defining enums for more things, but it did mean that I wasn't dealing with the commercially tested core that I thought I was. It also meant that I was a lot more concerned about a strange enemy lerping through the world bug I had seen a couple times.
I seriously considered going back to the virgin codebase and reimplementing the OpenGL rendering from scratch. The other thing that bothered me about the Redux codebase was that it was basically a graft of the Wolf3D code into the middle of a gutted Quake 2 codebase. This was cool in some ways, because it gave us a console, cvars, and the system / OpenGL portable framework, and it was clear the original intention was to move towards multiplayer functionality, but it was a lot of bloat. The original wolf code was only a few dozen C files, while the framework around it here was several times that.
Looking through the original code brought back some memories. I stopped signing code files years ago, but the top of WL_MAIN.C made me smile:
[code]/*
=============================================================================
WOLFENSTEIN 3-D
An Id Software production
by John Carmack
=============================================================================
*/ [/code]
It wasn't dated, but that would have been in 1991.
In the end, I decided to stick with the Redux codebase, but I got a lot more free with hacking big chunks of it out. I reimplemented load / save game (fixing the inevitable pointer bugs involved), and by littering asserts throughout the code, I tracked the other problem down to an issue with making a signed comparison against one of the new enum types that compare as unsigned. I'm still not positive if this was the right call, since the codebase is sort of a mess with lots of vestigial code that doesn't really do anything, and I don't have time to clean it all up right now.
Of course, someone else is welcome to do that. The full source code for the commercial app is available on the web site. There was a little thought given to the fact that if I had reverted to the virgin source, the project wouldn't be required to be under the GPL. Wolf and the app store presents a sort of unique situation -- a user can't just compile the code and choose not to pay for the app, because most users aren't registered developers, and the data isn't readily available, but there is actually some level of commercial risk in the fast-moving iPhone development community. It will not be hard to take the code that is already fun to play, pull a bunch of fun things off the net out of various projects people have done with the code over the years, dust off some old map editors, and load up with some modern quality art and sound.
Everyone is perfectly within their rights to go do that, and they can aggressively try to bury the original game if they want. However, I think there is actually a pretty good opportunity for cooperation. If anyone makes a quality product and links to the original Wolf app, we can start having links to "wolf derived" or "wolf related" projects.
That should turn out to be a win for everyone.
I'm going back to Rage for a while, but I do expect Classic Doom to come fairly soon for the iPhone.
-----------------------------------------
John Carmack's .plan for May 27, 2009
-----------------------------------------
iPhone development: Doom Classic Progress Report
Source: http://www.idsoftware.com/iphone-doom-classic-progress/
I have been spending the majority of my time working on iPhone Doom Classic for several weeks now, and the first beta build went out to some external testers a couple days ago. I am moving back on to Rage for a while, but I expect to be able to finish it up for submission to the App Store next month.
Wolfenstein 3D Classic was a quickie project to satisfy my curiosity and test the iPhone waters, but Doom is a more serious effort. In addition to the millions of people with fond memories of the game, there is still an active gaming / development community surrounding the original Doom, and I don't want to disappoint any of them.
One of the things I love about open sourcing the old games is that Doom has been ported to practically everything with a 32 bit processor, from toasters to supercomputers. We hear from a lot of companies that have moved the old games onto various set top boxes and PDAs, and want licenses to sell them. We generally come to some terms in the five figure range for obscure platforms, but it is always with a bit of a sigh. The game runs, and the demo playbacks look good, but there is a distinct lack of actually caring about the game play itself. Making Doom run on a new platform is only a couple days of work. Making it a really good game on a platform that doesn't have a keyboard and mouse or an excess of processing power is an honest development effort.
To my surprise, Christian was able to dig up the original high quality source material for the Doom sounds, so we have 22khz 16 bit sound effects instead of the 11khz 8 bit ones from the original game. It turns out that I can barely tell the difference, which is a sign that we made good choices way back then about catering the sounds to the output device. If we were on the fence for any resource limits, I would have considered sticking with the originals, but the current OpenAL mixer code has errors with 8 bit source buffers, so I would have had to convert to 16 bit at load time anyway, and just referencing the high quality source media actually speeds up the load times.
The music is all stored as mp3, performed on a high quality synthesizer. For Wolf, we used ogg, because that's what was in the Redux codebase that I started with, but I don't have all that CPU performance margin anymore, so it was necessary to use the iPhone's audio decompression hardware through the AudioQueue services. The music is the largest part of the application, but everything else is still well over the 10 meg cellular app transfer limit, so I'm not tempted to try and squeeze it under like we did with Wolfenstein. Maybe being able to get an app over 3G really isn't very important to its success. The fact that people are downloading Myst on the iPhone is heartening -- I have ideas for leveraging our high end idTech-5 content creation pipeline for a future iPhone game, if people will go for a few hundred meg download.
The toughest question was the artwork. Since Wolf was selling well, I had planned on paying contractors to upscale all the Doom graphics to twice the original resolution. When I pulled all the graphics out and tallied it all up, it looked a lot more marginal than I had expected. There were over two thousand individual pieces of art, and it was over ten megatexels in exactly bounded area, let alone atlas fit or power of two inset. The PVRTC compressed formats would work great for the floors and ceilings, which are all nice 64x64 blocks, but it has issues for both the walls and floors.
PVRTC textures must be power of two and, notably, square. If you want a 256 x 512 texture that needs to repeat in both axis, you need to resample it to 512 x 512 to use PVRTC, which means you lose half your compression benefit and get distorted filter kernels and mip level selections. Even worse, Doom had the concept of composited walls, where a surface was generated by adding, say, a switch texture on top of a wall texture. You can't do that sort of operation with PVRTC images. The clean sheet of paper solution to both problems is to design around textures that the hardware likes and use more geometry where you need to tile or combine them, but Doom wasn't well suited to that approach.
Character sprites don't get repeated, so a lot of them can be packed into a nice square 1024 x 1024 texture to minimize waste, but the PVRTC texture formats aren't well suited to sprite graphics. The DXT1 compression format has an exact bit mask for the alpha channel, which is what you want for an application like this. PVRTC treats alpha like any other color channel, so you get coupling between the alpha and color channels that results in partially transparent pixels ringing outside the desired character boundary. It works fine for things like clouds or fireballs, but not so good for character sprites. It looks like it should be possible to get an exact binary mask with the 2 bit PVRTC mode, which could be combined with a 4 bit PVRTC color texture to get a 6 bpp perfectly outlined sprite, but the multitexture performance on the iPhone, even with PVRTC textures, is not fast enough to prevent missing 30 fps when you have a hoard of monsters in front of you.
We started to do some internal samples of up-scaled artwork to use as reference for getting the contractor quotes, and it just wasn't looking all that spectacular. Doubling the art and smoothing out the edges wasn't buying much. There was certainly a lot of room for improvement, since Doom was designed around a 256 color palette with a limited selection of intensity ramps for lighting, but moving there from the starting point would be tricky. If I went to one of our artists today and asked them to draw a bad-ass Baron of Hell throwing a fireball in a 256 x 256 x 16 bit block, I would get something a LOT better than the original art, but it would look different, not just better.
I was also a little taken aback by some of the backlash against the updated graphics that I put in for Wolf 1.1. I took the walls, guns, and decorative sprites from the Mac version of Wolfenstein, and had Eric use that as source to recreate some similar graphics that weren't present in the Mac version. After release, there were a number of reviews that complained, saying that it "ruined the classic feel". I have a couple thoughts abut this: Changing the look on a point release is going to cause some level of complaint, so it is probably a good idea to make any changes from " classic" you think you might want in version 1.0. I also believe most of the complaints were due to the view weapons. The original gun artwork wasn't great, but the double-res ones weren't very good either, and they were a bit different looking. I debated with myself a bit about using them at all, and it looks like I probably shouldn't have. I can't see any drawback whatsoever to the double res walls and sprites, since they are in the same style, just better looking when you jam your face up against them.
In the end, I decided not to do anything with the DOOM source art. With the GPU accelerated filtering and 24 bit lighting it looks a lot better than it ever did, and with floors, ceilings, and lighting you don't seem to notice the low resolution as much as with Wolf.
With the speed (a solid 30 fps, even in the more aggressive later levels), the audio, the resolution, and the rendering quality, it is Doom as you remember it, which is quite a bit better than it actually was. If you go back and play the original game on a 386 with a sound blaster, you will be surprised at the 15 fps, FM-synth music, "bathroom tile sized" 320 x 200 pixels, external network game setup utility, and external keyboard configuration. A lot of people remember it as "The best game EVER!", but "ever" has sure moved a lot in the last decade!
Before I actually started coding on the project, I had visions of adding a lot of modern tuned effects to the core gameplay experience. It would certainly stay a sprite-and-sector based game, but there are many things that would be done differently with the benefit of a GPU and the wisdom of hindsight. Once I began actually working on it, it started to look like a bad idea for a number of reasons. I am trying to not be very disruptive in the main codebase, because I want it to stay a part of http://prboom.sourceforge.net/ instead of being another codebase fork. While I can certainly add a bunch of new features fairly quickly, iterating through a lot of user testing and checking for problems across the >100 commercial Doom levels would take a lot longer. There really is value in " classic" in this case, and there would be some degree of negative backlash to almost any "improvements" I made. There will still be a couple tiny tweaks, but nothing radical is changing in the basic play. It would be fun to take a small team, permanently fork it, and make a "Doom++" just for the iPhone, but that wouldn't be the best first move. Maybe later.
The iPhone interface around the game is all done nicely. Wolf Classic got dinged a bit for the blocky look of the buttons and interface components. I didn't actually see any complaints about the crappy monospace font, but it deserved some. Everything looks good now.
The initial release will be for OS 2.x, and support multiplayer over WiFi. A later release will be for 3.x only, and support bluetooth multiplayer. I looked into the possibility of 3G multiplayer, but the latencies just aren't good enough -- I see 380 or so pings from my phone to local servers. This was interesting, because I have talked to other hardware vendors that claim 3G latencies of half that. I'm not sure if there are issues with the iPhone, issues with AT&T's network in Dallas, or if the vendor was just mistaken about what they were getting. One anecdotal report is that iPhones work better in Japan than here, so it may be infrastructure.
I will probably have another update later with more technical details about the logic behind the new rendering architecture (rewritten for > 2x the speed of the original prBoom GL renderer), touch control issues, and so on.
-----------------------------------------
John Carmack's .plan for Nov 03, 2009
-----------------------------------------
iPhone development: Doom Classic
Source: http://www.idsoftware.com/doom-classic/doomdevelopment.htm
Way back in March when I released the source for Wolfenstein 3D Classic, I said that Doom Classic would be coming "real soon", and on April 27, I gave a progress report: http://www.idsoftware.com/iphone-doom-classic-progress/
I spent a while getting the multiplayer functionality up, and I figured I only had to spend a couple days more to polish things up for release.
However, we were finishing up the big iPhone Doom Resurrection project with Escalation Studios, and we didn't want to have two Doom games released right on top of each other, so I put Doom Classic aside for a while. After Doom Resurrection had its time in the sun, I was prepared to put the rest of the work into Doom Classic, but we ran into another schedule conflict. As I related in my Wolf Classic notes http://www.idsoftware.com/wolfenstein-3d-classic-platinum/wolfdevelopment.htm , Wolfenstein RPG for the iPhone was actually done before Wolfenstein Classic, but EA had decided to sit on it until the release of the big PC / console Wolfenstein game in August.
I really thought I was going to go back and finish things up in September, but I got crushingly busy on other fronts. In an odd little bit of serendipity, after re-immersing myself in the original Doom for the iPhone, I am now working downstairs at Id with the Doom 4 team. I'm calling my time a 50/50 split between Rage and Doom 4, but the stress doesn't divide. September was also the month that Armadillo Aerospace flew the level 2 Lunar Lander Challenge:
http://www.armadilloaerospace.com/n.x/Armadillo/Home/News?news_id=368
Finally, in October I SWORE I would finish it, and we aimed for a Halloween release. We got it submitted in plenty of time, but we ran into a couple approval hiccups that caused it to run to the very last minute. The first was someone incorrectly thinking that the "Demos" button that played back recorded demos from the game, was somehow providing demo content for other commercial products, which is prohibited. The second issue was the use of an iPhone image in the multiplayer button, which we had to make a last minute patch for.
@Release notes@
Ok, the game is finally out (the GPL source code is being packaged up for release today). Based on some review comments, there are a couple clarifications to be made:
Multiplayer requires a WiFi connection that doesn't have UDP port 14666 blocked. I'm quite happy with the simple and fast multiplayer setup, but it seems like many access points just dump the packets in the trash. If the multiplayer button on the main menu doesn't start pulsing for additional players after the first player has hit it, you won't be able to connect. I have also seen a network where the button would pulse, but the player would never get added to the player list, which meant that somehow the DNS packets were getting through, but the app packets weren't. It works fine on a normal AirPort install... More on networking below.
I took out tilt-to-turn just to free up some interface screen space, because I didn't know anyone that liked that mode, and my query thread on Touch Arcade didn't turn up people that would miss it a lot.
Evidently there are a few people that do care a lot, so we will cram that back in on the next update. The functionality is still there without a user interface, so you can enable it by four-finger-tapping to bring up the keyboard and typing "tiltturn 4000" or some number like that, and it will stay set. Make sure you have tiltmove pulled down to 0. I never got around to putting in a real console, but you can change a few parameters like that, as well as enter all the original doom cheat codes like IDDQD, IDKFA, etc.
I think that the auto-centering control sticks in Doom Classic are a better control scheme than the fixed sticks from Wolf Classic. The advice for wolf was to adjust the stick positions so that your thumbs naturally fell in the center point, so I just made that automatic for Doom. Effective control always involved sliding your thumbs on the screen, rather than discretely tapping it, and this mode forces you to do that from the beginning. Still, even if the new mode is some fraction "better", there are a lot of people who have logged a lot of hours in Wolfenstein Classic, and any change at all will be a negative initially. In the options->settings menu screen, there is a button labeled "Center sticks: ON" that can be toggled off to keep the sticks fixed in place like in Wolf.
A subtle difference is that the turning sensitivity is now graded so that a given small movement will result in a specific percentage increase in speed, no matter where in the movement range it is. With linear sensitivity, if you are 10 pixels off from the center and you move your thumb 10 pixels farther, then the speed exactly doubles. If you are 50 pixels off from the center, the same 10 pixel move only increases your turning rate by 20%. With ramped sensitivity, you would get a 20% (depending on the sensitivity scale) increase in speed in both cases, which tends to be better for most people. You can disable this by toggling the "Ramp turn: ON" option off.
In hindsight, I should have had a nice obvious button on the main options screen that said "Wolfenstein Style" and had the same options, but I have always had difficult motivating myself to do good backwards compatibility engineering. Even then, the movement speeds are different between the games, so it wouldn't have felt exactly the same.
It was a lot of fun to do this project, working on it essentially alone, as a contrast to the big teams on the major internal projects. I was still quite pleased with how the look and feel of the game holds up after so long, especially the "base style" levels. The "hell levels" show their age a lot more, where the designers were really reaching beyond what the technology could effectively provide.
@Future iPhone work@
We do read all the reviews in the App store, and we do plan on supporting Doom Classic with updates. Everything is still an experiment for us on the iPhone, and we are learning lessons with each product. At this point, we do not plan on making free lite versions of future products, since we didn't notice anything worth the effort with Wolfenstein, and other developers have reported similar findings.
We have two people at Id that are going to be dedicated to iPhone work. I doubt I will be able to personally open Xcode again for a few months, but I do plan on trying to work out a good touch interface for Quake Classic and the later 6DOF games. I also very much want to make at least a tech demo that can run media created with a version of our idTech 5 megatexture content creation pipeline. I'm not sure exactly what game I would like to do with it, so it might be a 500 mb free gee-whiz app...
Wolfenstein Classic Platinum was a break-in opportunity for the new internal iPhone developers. We were originally planning on making the Spear of Destiny levels available as in-app purchased content. Then we decided to make it a separate "Platinum Edition" application at a reasonable price. Finally, we decided that we would just make it a free update, but something has gone wrong during this process -- people who buy the app for the first time get everything working properly, but many people who upgrade the App from a previous purchase are seeing lots of things horribly broken. We are working with Apple to try to debug and fix this, but the workaround is to uninstall the app completely, then reload it. The exciting thing about Wolf Platinum is the support for downloadable levels, which is the beta test for future game capabilities. Using a URL to specify downloadable content for apps is a very clean way to interface to the game through a web page or email message.
The idMobile team is finishing up the last of the BREW versions of Doom 2 RPG, and work has started on an iPhone specific version, similar to the Wolfenstein RPG release. The real-time FPS games are never going to be enjoyable for a lot of people, and the turn based RPG games are pretty neat in many regards. If they are well received, we will probably bring over the Orcs&Elves games as well.
I want to work on a Rage themed game to coincide with Rage's release, but we don't have a firm direction or team chosen for it. I was very excited about doing a really-designed-for-the-iPhone first person shooter, but at this point I am positive that I don't have the time available for it.
@Networking techie stuff@
I doubt one customer in ten will actually play a network game of Doom Classic, but it was interesting working on it.
Way back in March when I was first starting the work, I didn't want the game to require 3.0 to run, and I generally try to work with the lowest level interfaces possible for performance critical systems, so I wasn't looking at GameKit for multiplayer. I was hoping that it was possible to use BSD sockets to allow both WiFi networking on 2.0 devices and WiFi or ad-hoc bluetooth on 3.0 devices. It turns out that it is possible, but it wasn't documented as such anywhere I could find.
I very much approve of Apple's strategy of layering Obj-C frameworks on top of Unix style C interfaces. Bonjour is a layer over DNS, and GameKit uses sockets internally. The only bit of obscure magic that goes on is that the bluetooth IP interface only comes into existence after you have asked DNS to resolve a service that was reported for it. Given this, there is no getting around using DNS for initial setup.
With WiFi, you could still use your own broadcast packets to do player finding and stay completely within the base sockets interfaces, and this might even make some sense, considering that there appear to be some WiFi access points that will report a DNS service's existence that your app can't actually talk to.
For every platform I have done networking on previously, you could pretty much just assume that you had the loopback interface and an Ethernet interface, and you could just use INADDR_ANY for pretty much everything. Multiple interfaces used to just be an issue for big servers, but the iPhone can have a lot of active interfaces -- loopback, WiFi Ethernet, Bluetooth Ethernet, and several point to point interfaces for the cellular data networks.
At first, I was excited about the possibility of multiplayer over 3G. I had been told by someone at Intel that they were seeing ping times of 180 ms on 3G devices, which could certainly be made to work for gaming.
Unfortunately, my tests, here in Dallas at least, show about twice that, which isn't worth fighting. I'm a bit curious whether they were mistaking one-way times, or if the infrastructure in California is really that much better. In any case, that made my implementation choice clear -- local link networking only.
A historical curiosity: the very first release of the original Doom game on the PC used broadcast IPX packets for LAN networking. This seemed logical, because broadcast packets for a network game of N players has a packet count of just N packets on the network each tic, since everyone hears each packet. The night after we released the game, I was woken up by a call from a college sysadmin yelling at me for crippling their entire network. I didn't have an unlisted number at the time. When I had decided to implement network gaming, I bought and read a few books, but I didn't have any practical experience, so I had thought that large networks were done like the books explained, with routers connecting independent segments. I had no idea that there were many networks with thousands of nodes connected solely by bridges that forwarded all broadcast packets over lower bit rate links. I quickly changed the networking to have each peer send addressed packets to the other peers. More traffic on the local segment, but no chance of doing horrible things to bridged networks.
WiFi is different from wired Ethernet in a few ways. WiFi clients don't actually talk directly to each other, they talk to the access point, which rebroadcasts the packet to the destination, so every packet sent between two WiFi devices is actually at least two packets over the air.
An ad-hoc WiFi network would have twice the available peer to peer bandwidth and half the packet drop rate that an access point based one does. Another point is that unlike wired Ethernet, the WiFi link level actually does packet retransmits if the destination doesn't acknowledge receipt. They won't be retransmitted forever, and the buffer spaces are limited, so it can't be relied upon the way you do TCP, but packet drops are more rare than you would expect. This also means that there are lots of tiny little ACK packets flying around, which contributes to reduced throughput. Broadcast packets are in-between -- the broadcast packet is sent from the source to the access point with acknowledgment and retransmits, but since the access point can't know who it is going to, it just fires it out blind a single time.
I experimentally brought the iPhone networking up initially using UDP broadcast packets, but the delivery was incredibly poor. Very few packets were dropped, but hundreds of milliseconds could sometimes go by with no deliveries, then a huge batch would be delivered all at once. I thought it might be a policy decision on our congested office access point, but it behaved the same way at my house on a quiet LAN, so I suspect there is an iPhone system software issue. If I had a bit more time, I would have done comparisons with a WiFi laptop. I had pretty much planned to use addressed packets anyway, but the broadcast behavior was interesting.
Doom PC was truly peer to peer, and each client transmitted to every other client, for N * (N-1) packets every tic. It also stalled until valid data had arrived from every other player, so adding more players hurts in two different ways -- more packets = more congestion = more likely to drop each individual packet. The plus side of an arrangement like this is that it is truly fair, no client has any advantage over any other, even if one or more players are connected by a lower quality link. Everyone gets the worst common denominator behavior.
I settled on a packet server approach for the iPhone, since someone had to be designated a "server" anyway for DNS discovery, and it has the minimum non-broadcast packet count of 2N packets every tic. Each client sends a command packet to the server each tic, the server combines all of them, then sends an addressed packet back to each client. The remaining question was what the server should do when it hasn't received an updated command from a client. When the server refused to send out a packet until it had received data from all clients, there was a lot more variability in the arrival rate. It could be masked by intentionally adding some latency on each client side, but I found that it plays better to just have the server repeat the last valid command when it hasn't gotten an update. This does mean that there is a slight performance advantage to being the server, because you will never drop an internal packet.
The client always stalls until it receives a server packet, there was no way I had the time to develop any latency reducing / drop mitigating prediction mechanisms. There are a couple full client / server, internet capable versions of Doom available on the PC, but I wanted to work from a more traditional codebase for this project.
So, I had the game playing well over WiFi, but communication over the Bluetooth interface was significantly worse. There was an entire frame of additional latency versus WiFi, and the user mode Bluetooth daemon was also sucking up 10% of the CPU. That would have been livable, but there were regular surges in the packet delivery rate that made it basically unplayable.
Surging implies a buffer somewhere backing up and then draining, and I had seen something similar but less damaging occasionally on WiFi as well, so I wondered if there might be some iPhone system communication going on. I spent a little while with WireShark trying to see if the occasional latency pileup was due to actual congestion, and what was in the packets, but I couldn't get my Macbook into promiscuous WiFi mode, and I didn't have the time to configure a completely new system.
In the end, I decided to just cut out the Bluetooth networking and leave it with WiFi. There was a geek-neatness to having a net game with one client on WiFi and another on Bluetooth, but I wasn't going to have time to wring it all out.
-----------------------------------------
John Carmack's .plan for Oct 26, 2010
-----------------------------------------
RAGE on iPhone/iPad/iPod
Source: http://www.bethblog.com/index.php/2010/10/29/john-carmack-discusses-rage-on-iphoneipadipod-touch/
@RAGE for iPhone@
Our mobile development efforts at id took some twists and turns in the last year. The plan was always to do something RAGE-related on the iPhone/iPad/iPod touch next, but with all the big things going on at id, the mobile efforts werenít front and center on the priority list. There had been a bit of background work going on, but it was only towards the end of July that I was able to sit down and write the core engine code that would drive the project.
I was excited about how well it turned out, and since this was right before QuakeCon, I broke with tradition and did a live technology demo during my keynote. In hindsight, I probably introduced it poorly. I said something like ìIts RAGE. On the iPhone. At 60 frames a second.î Some people took that to mean that the entire PC/console game experience was going to be on the iPhone, which is definitely not the case.
What I showed was a technology demo, written from scratch, but using the RAGE content creation pipeline and media. We do not have the full RAGE game running on iOS, and we do not plan to try. While it would (amazingly!) actually be possible to compile the full-blown PC/console RAGE game for an iPhone4 with some effort, it would be a hopelessly bad idea. Even the latest and greatest mobile devices are still a fraction of the power of a 360 or PS3, let alone a high end gaming PC, so none of the carefully made performance tradeoffs would be appropriate for the platform, to say nothing of the vast differences in controls.
What we do have is something unlike anything ever seen on the iOS platforms. It is glorious, and a lot of fun. Development has been proceeding at high intensity since QuakeCon, and we hope to have the app out by the end of November.
The technical decision to use our megatexture content creation pipeline for the game levels had consequences for its scope. The data required for the game is big. Really, really big. Seeing Myst do well on the iPhone with a 700 meg download gave me some confidence that users would still download huge apps, and that became the target size for our standard definition version, but the high definition version for iPad / iPhone 4 will be around twice that size. This is more like getting a movie than an app, so be prepared for a long download. Still, for perspective, the full scale RAGE game is around 20 gigs of data with JPEG-XR compression, so 0.7 gigs of non-transcoded data is obviously a tiny slice of it.
Since we werenít going to be able to have lots of hugely expansive levels, we knew that there would be some disappointment if we went out at a high price point, no matter how good it looked. We have experimented with a range of price points on the iPhone titles so far, but we had avoided the very low end. We decided that this would be a good opportunity to try a $0.99 SD / $1.99 HD price point. We need to stay focused on not letting the project creep out of control, but I think people will be very happy with the value.
The little slice of RAGE that we decided to build the iPhone product around is ìMutant Bash TVî, a post apocalyptic combat game show in the RAGE wasteland. This is the perfect setup for a quintessential first person shooter game play experience ó you pick your targets, aim your shots, time your reloads, dodge the bad guys, and try and make it through to the end of the level with a better score than last time. Beyond basic survival, there are pickups, head shots, and hit streak multipliers to add more options to the gameplay, and there is a broad range of skill levels available from keep-hitting-fire-and-you-should-make-it to almost-impossible.
A large goal of the project has been to make sure that the levels can be replayed many times. The key is making the gamplay itself the rewarding aspect, rather than story progression, character development, or any kind of surprises. Many of the elements that made Doom Resurrection good the first time you played it hurt the replayability, for instance. RAGE iOS is all action, all the time. I have played the game dozens of times, and testing it is still fun instead of a chore.
@Technical Geek Details@
The id Tech 5 engine uses a uniform paged virtual texture system for basically everything in the game. While the algorithm would be possible on 3GS and later devices, it has a substantial per-fragment processing cost, and updating individual pages in a physical texture is not possible with PVRTC format textures. The approach used for mobile RAGE is to do the texture streaming based on variable sized contiguous ìtexture islandsî in the world. This is much faster, but it forces geometric subdivision of large surfaces, and must be completely predictive instead of feedback reactive. Characters, items, and UI are traditionally textured.
We build the levels and preview them in RAGE on the PC, then run a profiling / extraction tool to generate the map data for the iOS game. This tool takes the path through the game and determines which texture islands are going to be visible, and at what resolution and orientation. The pixels for the texture island are extracted from the big RAGE page file, then anisotropically filtered into as many different versions as needed, and packed into 1024◊1024 textures that are PVRTC compressed for the device.
The packing into the textures has conflicting goals ñ to minimize total app size you want to cram texture islands in everywhere they can fit, but you also donít want to scatter the islands needed for a given view into a hundred different textures, or radically change your working set in nearby views. As with many NP complete problems, I wound up with a greedy value metric optimizing allocation strategy.
Managing over a gig of media made dealing with flash memory IO and process memory management very important, and I did a lot of performance investigations to figure things out.
Critically, almost all of the data is static, and can be freely discarded. iOS does not have a swapfile, so if you use too much dynamic memory, the OS gives you a warning or two, then kills your process. The bane of iOS developers is that ìtoo muchî is not defined, and in fact varies based on what other apps (Safari, Mail, iPod, etc) that are in memory have done. If you read all your game data into memory, the OS canít do anything with it, and you are in danger. However, if all of your data is in a read-only memory mapped file, the OS can throw it out at will. This will cause a game hitch when you need it next, but it beats an abrupt termination. The low memory warning does still cause the frame rate to go to hell for a couple seconds as all the other apps try to discard things, even if the game doesnít do much.
Interestingly, you can only memory map about 700 megs of virtual address space, which is a bit surprising for a 32 bit OS. I expected at least twice that, if not close to 3 gigs. We sometimes have a decent fraction of this mapped.
A page fault to a memory mapped file takes between 1.8 ms on an iPhone 4 and 2.2 ms on an iPod 2, and brings in 32k of data. There appears to be an optimization where if you fault at the very beginning of a file, it brings in 128k instead of 32k, which has implications for file headers.
I am pleased to report that fcntl( fd, F_NOCACHE ) works exactly as desired on iOS ñ I always worry about behavior of historic unix flags on Apple OSs. Using this and page aligned target memory will bypass the file cache and give very repeatable performance ranging from the page fault bandwidth with 32k reads up to 30 mb/s for one meg reads (22 mb/s for the old iPod). This is fractionally faster than straight reads due to the zero copy, but the important point is that it wonít evict any other buffer data that may have better temporal locality. All the world megatexture data is managed with uncached reads, since I know what I need well ahead of time, and there is a clear case for eviction. When you are past a given area, those unique textures wonít be needed again, unlike, say monster animations and audio, which are likely to reappear later.
I pre-touch the relevant world geometry in the uncached read thread after a texture read has completed, but in hindsight I should have bundled the world geometry directly with the textures and also gotten that with uncached reads.
OpenAL appears to have a limit of 1024 sound buffers, which we bumped into. We could dynamically create and destroy the static buffer mappings without too much trouble, but that is a reasonable number for us to stay under.
Another behavior of OpenAL that surprised me was finding (by looking at the disassembly) that it touches every 4k of the buffer on a Play() command. This makes some sense, forcing it to page the entire thing into ram so you donít get broken sound mixing, but it does unpredictably stall the thread issuing the call. I had sort of hoped that they were just eating the page faults in the mixing thread with a decent sized mix ahead buffer, but I presume that they found pathological cases of a dozen sound buffers faulting while the GPU is sucking up all the bus bandwidth or some such. I may yet queue all OpenAL commands to a separate thread, so if it has to page stuff in, the audio will just be slightly delayed instead of hitching the framerate.
I wish I could prioritize the queuing of flash reads ñ game thread CPU faults highest, sound samples medium, and textures lowest. I did find that breaking the big texture reads up into chunks helped with the worst case CPU stalls.
There are two project technical decisions that I fretted over a lot:
Because I knew that the basic rendering technology could be expressed with fixed function rendering, the game is written to OpenGL ES 1.1, and can run on the older MBX GPU platforms. While it is nice to support older platforms, all evidence is that they are a negligible part of the market, and I did give up some optimization and feature opportunities for the decision.
It was sort of fun to dust off the old fixed function puzzle skills. For instance, getting monochrome dynamic lighting on top of the DOT3 normal mapping in a single pass involved sticking the lighting factor in the alpha channel of the texture environment color so it feeds through to the blender, where a GL_SRC_ALPHA, GL_ZERO blend mode effects the modulation on the opaque characters. This sort of fixed function trickery still makes me smile a bit, but it isnít a relevant skill in the modern world of fragment shaders.
The other big one is the codebase lineage.
My personally written set of iPhone code includes the renderer for Wolfenstein RPG, all the iPhone specific code in Wolfenstein Classic and Doom Classic, and a few one-off test applications. At this point, I feel that I have a pretty good idea of what The Right Thing To Do on the platform is, but I donít have a mature expression of that in a full game. There is some decent code in Doom Classic, but it is all C, and I would prefer to do new game development in (restrained) C++.
What we did have was Doom Resurrection, which was developed for us by Escalation Studios, with only a few pointers here and there from me. The play style was a pretty close match (there is much more freedom to look around in RAGE), so it seemed like a sensible thing. This fits with the school of thought that says ìnever throw away the codeî (http://www.joelonsoftware.com/articles/fog0000000069.html ). I take issue with various parts of that, and much of my success over the years has involved wadding things up and throwing it all away, but there is still some wisdom there.
I have a good idea what the codebase would look like if I wrote it from scratch. It would have under 100k of mutable CPU data, there wouldnít be a resource related character string in sight, and it would run at 60 fps on new platforms / 30 fps on old ones. Iím sure I could do it in four months or so (but I am probably wrong). Unfortunately, I canít put four months into an iPhone project. Iím pushing it with two months ñ I have the final big RAGE crunch and forward looking R&D to get back to.
So we built on the Resurrection codebase, which traded expediency for various compromise in code efficiency. It was an interesting experience for me, since almost all the code that I normally deal with has my ìcoding DNAî on it, because the id Software coding standards were basically ìprogram the way John does.î The Escalation programmers come from a completely different background, and the codebase is all STL this, boost that, fill-up-the-property list, dispatch the event, and delegate that.
I had been harboring some suspicions that our big codebases might benefit from the application of some more of the various ìmodernî C++ design patterns, despite seeing other large game codebases suffer under them. I have since recanted that suspicion.
I whine a lot about it (occasionally on twitter), and I sometimes point out various object lessons to the other mobile programmers, but in the end, it works, and it was probably the right decision.
John Carmack's .plan Archive in one jumbotweet!
(Covers 1996 -> 2010)
This was grabbed from http://floodyberry.com/carmack/plan.html, so thanks for making it available!
-----------------------------------------
John Carmack's .plan for Feb 18, 1996
-----------------------------------------
* page flip crap
* stretch console
* faster swimming speed
* damage direction protocol
* armor color flash
* gib death
* grenade tweaking
* brightened alias models
* nail gun lag
* dedicated server quit at game end
+ scoreboard
+ optional full size
+ view centering key
+ vid mode 15 crap
+ change ammo box on sbar
+ allow "restart" after a program error
+ respawn blood trail?
+ -1 ammo value on rockets
+ light up characters
vsync on high framerates
-----------------------------------------
John Carmack's .plan for Feb 19, 1996
-----------------------------------------
* fixed vanished bmodels at startup
* fixed getting stuck problem
* fixed the sound out of range problem
* hack patched the dissapearing under water
+ remove look spring
+ fix radius damage
+ too many people on status bar
+ invis in water
+ multiple pain sounds started
+ shift jumps while in console
+ less blood time
+ gone armor icon
+ look up/down keyboard modifier
clamp velocity on respawn
clear damage flash on respawn
bad changeup
bob speed kept on restart
no puffs in sky volumes
id copyright seal
teleport screen warp
-----------------------------------------
John Carmack's .plan for Feb 20, 1996
-----------------------------------------
* rewrote sound packet communication
* fixed multiple pains from a single shotgun
* qcc -copy option for sounds and models
* fixed up pak file support
* removed .rc files
* screen size over status bar
+ monsters need to be pushed into teleporters
+ blue frag pieces
+ NaN impact time is a crasher
+ looking down on weapon bounces away from bottom
+ timing preventing attacks at startup
fix hardwired vid_uopdate
remove redundant death sounds
teleport warp
torches
cpuid?
monsterblock entity
multiple sky textures in a level
-----------------------------------------
John Carmack's .plan for Feb 21, 1996
-----------------------------------------
* frag bar
* start profile on late frame
* fixed relinking bug
* improved prediction bounding
* change weapon chooses supers first
* ammo pictures on status bar
* drop pain flash again and bound
* tweak armor and spike damages
+ don't pick up equal armor
+ ceiling doors don't cause damage
+ demons crash when jumping
+ monsters need to use bsp clipping
+ need to save memory...
+ Fatal error: NaN impact time from entity
+ change armor damage
+ drop weapons
+ face to face with nailgun does no damage
+ +pointfile from cmdline doesn't work
+ conglomerate hunk printing
+ shoot triggers
+ missing a Z update on rockets sometimes?
+ firing after net games
+ negative scores on frag bar and frag board
+ world backpack and armor model
+ weapon bounce down
+ center look key
+ check change levels in high res mode
show ammo counts in single player frag screen
radius damage should open secret doors
five point font on kills
rocket poke into walls
shoot open connected doors
gib bounce sound
-----------------------------------------
John Carmack's .plan for Feb 22, 1996
-----------------------------------------
* fix incorrectly updated origin bug
* added alias backpack and armor
* fixed weapons sticking into wall
* changed "connected to server" to "connected to map"
* fixed console
* armor skin numbers
* fixed "not shooting" bug
* changed sound back to origin based instead of entity based
+ swimming directions
+ quit scores are wrong
+ screen flicker crap
+ radius damage sight check
+ crushing issues
+ bodies stopping in midair
dimmed out weapons
follow killer
spin head inside gib
-----------------------------------------
John Carmack's .plan for Feb 23, 1996
-----------------------------------------
* fixed sound origins again
* fixed crushing stuff
* added "wait" to triggers
* added startup stuff for demo
* swimming direction stuff
* added '+' sizeup as well as =
* changed "unnamed" to "player"
* fixed skinnum clearing
+ gibbs need to fall after trap doored
+ ambient
+ make bubble sound while swimming forward
+ server speed
+ case insensitive on key names
+ hunk alloc
+ nans
+ better notify scrolling
+ final score dumping
+ negative frags
+ reconnect problems
backpack dropping underlava
"hostname" makes gane think it is a server
swimming models
weapon cycling
dedicated server stuffcmds
-----------------------------------------
John Carmack's .plan for Feb 24, 1996
-----------------------------------------
* added case insensitive key names
* skipped license check on dedicated servers
* changed prog score sorting some
* took out autofire
* fixed negative frags
* cleaned up connection management
* fixed hunk_alloc
* no sutch alias frame problem when leaving server
* "map test1" on system with enough memory
* flickering side crap when level restarting
* added base directory to path
* changed heads to tosstop
****** RELEASE QTEST1 ******
+ unupdated Z problem is still in there
+ flickering weapon icon
+ blood trail to respawn
-----------------------------------------
John Carmack's .plan for Feb 25, 1996
-----------------------------------------
* emit bsp files
+ idle loop on disconnect
+ key sticking
+ fix noclip in the water
+ modelgen scale option
+ run a single level for a long time, then connect hangs
+ grenade radius check
gibs stuck in walls in low G
flash blinding
turbulent texture direction
space in smiley talk message
talk sounds
full chat mode
-----------------------------------------
John Carmack's .plan for Feb 26, 1996
-----------------------------------------
* ogre sounds
* made 180 turns go left for better ogre dragging
* error checked and extended qbsp
+ drop green slime percentage
+ ram chip overlapps turtle
+ do DEC quote
+ nolerp update
caps-lock = full speed
no particle effects on sky
-----------------------------------------
John Carmack's .plan for Feb 27, 1996
-----------------------------------------
* expand qbsp to 32 bit values for edges and faces. sigh.
* fixed key sticking problem
seperate mouse sscaling values to allow negative up/down reversal
frag bar shows first two digits, not last two when >= 100
randomize head direction
slime less damage
drop to knees when crunched
check deathmatch flags before precaching
-----------------------------------------
John Carmack's .plan for Feb 28, 1996
-----------------------------------------
+ upper health bound
+ keyboard freelook
+ more air acceleration
can a hostile monster hit it's movetarget and turn away? check scirk and smotte don't cache player model in single player "fair" mode with syncronous clients kill timers on all spawned objects
-----------------------------------------
John Carmack's .plan for Feb 29, 1996
-----------------------------------------
* increased air acceleration
+ don't allow up/down move at all without noclip
+ weird fall out of level in sbad
-----------------------------------------
John Carmack's .plan for Mar 01, 1996
-----------------------------------------
* added ping command
* fixed time warp from overloaded dedicated servers
* rotate binaries
* respawning items
+ clean model flags on remove
+ tim's color
+ weapon ammo not redrawn sometimes
+ no sutch alias frame after gibbing
+ give more feedback during the connection process
+ set absolute screen size command
safe allocation failures from vid / zbuffer
send full pathnames from server to client, not findfile
make all commands remote if priveledged
unique names
inquire about server rules (teamplay, fraglimit, timelimit,
maxspeed, gravity, ticrate)
is vid still flushing the entire screen?
empty path causes error in quake?
don't need to update frame of target entity
-----------------------------------------
John Carmack's .plan for Mar 02, 1996
-----------------------------------------
* wrote sgi test demo
-----------------------------------------
John Carmack's .plan for Mar 03, 1996
-----------------------------------------
* added precache_file qcc option for -copy files
* capped health at 250
+ solid, invisible brushes
+ don't send linear entities in signon message
+ write a message trace printing mode
+ is 256 angle quantization from client hurting us?
+ sky animations use the wrong time base
+ centralize the connection process
weapon switching is wrong
no twitch in god mode
if a linear update entity doesn't get an update in a packet, remove it
bound upper client message sends properly
RELEASE: bump protocol version numbers just before releasing
-----------------------------------------
John Carmack's .plan for Mar 04, 1996
-----------------------------------------
* got new internet code running
* server protocol number
* stopped printing name change message for first message
* put usleep back in
+ an entity model is being cleared out from underneath it
+ rotating body parts
+ binary bit fields
+ damage in datagrams
+ push back at edges when moving slow?
+ twirling gibs
+ bit fields for binary objects
+ never reuse binary object slots
+ fix connect to a bad map
+ noticed water to dm only clipping prob in tim13
+ no sutch alias frame problem
magic8ball.rogue-ent.com = 38.252.190.8
ability to make custom plat activation triggers
-----------------------------------------
John Carmack's .plan for Mar 05, 1996
-----------------------------------------
* basic zombie actions
* added dog
* fixed bug in QuakeEd that caused a crash when setting a bad classname
* changed health stat to a short to fix negative numbers
* added binary object bit update
+ zonbies need to change size when down
+ secret doors need to crush damage, and return to open
negative health
fish have problems
cycle general updates in wasted badnwidth
-----------------------------------------
John Carmack's .plan for Mar 06, 1996
-----------------------------------------
* fixed all network play issues
* better monster combat
* monsters don't activate plats
* shambler shoot up
* stubbed enforcer
* soldier standing frames
* monster use function makes them hostile towards activator
+ drop ammo
+ monsters see farther
+ negative ammo
+ extra friction near edges
+ fracs aren't right in single player
+ use monsters = wake up
set ideal yaw upon first sighting
temp object updates
check size on teleport destinations
player only teleporters
puff of smoke when shooters fire
weird high look up on romero's level
step on grenades
flyers look down
eagle eye checkbox
monsters not trigger plat
flying change height
monster jumping
make tents for shotgun puffs
leave level running too long problem
shambler range
-----------------------------------------
John Carmack's .plan for Mar 07, 1996
-----------------------------------------
* increased max portals on edge in bsper
* NT gl testing
* fixed path walking
+ clear static objects for restarts
-----------------------------------------
John Carmack's .plan for Mar 08, 1996
-----------------------------------------
* dropping ammo
* zombie gib non-solid
* all zombie attacks
* scrolling notify text
* shooters that fire superspikes
* keys
* increased monster sight distance
* fixed noclip in water
+ fix sound code
+ item only in dm flags
+ shoot through complex bmodels
+ NaN stuff
+ change impulse over to impule # instead of impulse1
+ dog frames
+ demos!
fix noclip underwater
all zombie pain issues
duplicate impulse events
status bar art needs work
jrbase3 teleporter can be hit behind dmonly
do pain well on all monsters
don't blit the entire screen when sized down!
clean up headers
-----------------------------------------
John Carmack's .plan for Mar 09, 1996
-----------------------------------------
* wrote cl_shownet 2 dump
* rebuilt qcc to take $scale parm
* worked on demon
* cleaned up obsolete utilites
* fixed super shotgun size for sbad
* moved wadfile to hunk
* added colormap grab to qlumpy and got rid of vid_256.h
* finaly gave dec quote
* spread monster thinking times
+ fix angle on spike shooters
+ pick up super nailgun == illegible server message?
+ damaging trains
+ sandy: sally, don't put all the weapons on it, and help the PVS at start
+ get rid of surface->drawnframe now?
fix weapon switching
fix sound mixing
climb out of water
weapon pickup can overmax ammo
staticly define sound arrays
lightning client entity
fly move to goal
checkbox
stop repeat jump
check DAMAGE_YES on dead bodies
func_oscilating
teleport warp
is surface->texmultab[MIPLEVELS] worth the space?
combine surface->texture and ofs into texinfo?
-----------------------------------------
John Carmack's .plan for Mar 10, 1996
-----------------------------------------
* fixed static object update bomb
* fixed crazy speed guys
* allow restart after objerror
* fixed bug with restarting droped out of world messages
* spread standing frames more for groups
+ remove precache_item
+ ogres that won't wake up
+ spawn flags
pause turn speed
army guys don't wake up by touch?
jump velocity shouldn't make head bob
-----------------------------------------
John Carmack's .plan for Mar 11, 1996
-----------------------------------------
* fixed backpack dropping in deathmatch
* fixed monster sighting PVS problem
+ fix dos qcc
+ demon landing frame
+ trigger activated train
+ enforcer
+ dog
+ -1 ammo, 1 ammo on super shotgun, etc
+ invisible brushes for clip hull
+ grenade launcher shouldn't aim move
update ammo when picked up grenade launcher with rocket running
damage value on trains and secret doors
player stayed on console after leaving
jctest1 door fuckups
grenade open secret doors
-----------------------------------------
John Carmack's .plan for Mar 12, 1996
-----------------------------------------
* got new alias stuff sort of working
+ pause message needs to stay up
+ change level trigger
+ knight bodies are solid
lighting hidden objects
level loading pacifier
weapon switch button
doors that stay shut on some multi levels?
mouse button stuck down
monster shouldn't get key door messages
-----------------------------------------
John Carmack's .plan for Mar 13, 1996
-----------------------------------------
* got new alias stuff (mostly) debugged
* optimized BoxOnPlaneSide
* stoped generating backwards planes
* demon jump higher, fixed melee attacks
* made knight corpse non-solid
* crunched progs.dat file structure
* culled unused edict fields
+ shoot trigger
+ alias clipping still isn't right
+ reverse dmonly / not_dm? combine with spawn only flags?
+ work on sound code!
+ fixed secret doors stop working
+ make objects fall when floor is removed
make #ifdef PARANOID checks
lightweight entities to save space?
grenade / rocket damage to triggers
-----------------------------------------
John Carmack's .plan for Mar 14, 1996
-----------------------------------------
* new trigger code with delay
+ jrbase3 spawn problem
jump triggers
dragon path
uncache player on single player
monster missile checking for race relations
gib pack
wizard partical trail
multiple sky maps
non-bleeding takedamage
-----------------------------------------
John Carmack's .plan for Mar 15, 1996
-----------------------------------------
* made an execution stack for progs
* spawn flags
+ fix NaNs!
+ proper level changing
door no backup option
PVS for net update
check entity relinking for excess
good tempents.
-----------------------------------------
John Carmack's .plan for Mar 16, 1996
-----------------------------------------
* made -1 wait doors not return when blocked
* made secret doors and trains do damage
* added edge friction
* fixed reset self bug that caused jrbase3 not to load
* loading pacifiers
* split out some of the physics code
+ do from-the-beginning demos
+ why are NeXT fullbright colors still bad?
+ not starting up at american's house?
+ move the if(deathmatch) remove (self) statements before monster
precaches
+ trigger start trains
shambler and wizard need idle frames
bump all version numbers before final release, so no maps work with
qtests!
general sound objects?
mouse button sticking
win95 startup with map
eliminate MAX_FACES from qbsp? (johncm1.map)
-----------------------------------------
John Carmack's .plan for Mar 17, 1996
-----------------------------------------
* triggered trains
* began work on sound code
+ edge friction jumping
+ edge friction in air?
ambush guards
temp ent precache
does pacifier not work on net games?
-----------------------------------------
John Carmack's .plan for Mar 18, 1996
-----------------------------------------
* fuxed button triggered delays
* added classname to light error message
* demon more damage
* sounder objects
* light_switchable
+ grenade proximity explosions
+ fix jumping off edges
+ wizard fast shots
+ target lights for controling them
+ plat hit on head bug
+ ambient light
spiral the view when gibbed
level based gravity
-----------------------------------------
John Carmack's .plan for Mar 19, 1996
-----------------------------------------
* switchable lights
+ changed light program to unique switchable styles
+ target usable monsters
+ toggleable doors
+ leaf lighting
+ F2 reload crashes
+ grenade radius
+ spikeshooter directions
+ sound cache shouldn't be cleared all the times
+ demon jumping
+ demon death frames
+ chain activators for monster deaths
+ crawl out of water
+ ambient from map
autosavemap?
different teleport flash
linked doors still have problems
work on swimming
control triggers for player only, or with monsters
better knight code
turn off regular shooters when used?
once trigger on door_start_up?
ctrl-alt-delete reset?
remove r_fullbright in deathmatch
sound streaming
fix "couldn't allocate Z buffer" error
reduce sound latency
-----------------------------------------
John Carmack's .plan for Mar 20, 1996
-----------------------------------------
* deathmatch onyl spawn doesn't work
* drowning damage
* changed QuakeEd to get progs from id1_
* added colors to QuakeEd entities
* got rid of /lquake, moved QuakeEd caches to /qcache
+ don't take datagrams until acked second message
+ not-dm is intangible
+ sally torches not spawning
+ client called twice?
+ combine damage messages from super shotguns
+ gib zombies lying down?
+ more water friction
+ swimming bubble noise
+ light the weapons differently in bright areas
+ keyboard freelook
+ move QuakeEd start position to playerstart
stop wall climbing
sleeping pause
raise rocket launcher with 0 ammo
-----------------------------------------
John Carmack's .plan for Mar 21, 1996
-----------------------------------------
* fixed quakeed bug to allow carrying selection to new map
* fixed updated static bug
+ demons that fall through world
+ "wait" exec command
+ end of line exec bug
do style lighting for alias models
for release: Id Trademark graphic
-----------------------------------------
John Carmack's .plan for Mar 22, 1996
-----------------------------------------
* fixed bug with "alias" command that failed to add a newline
* fixed bug in command execution that sometimes mashed commands together
* added "impulse #" command to replace impulse0 impulse1, etc
* allowed "bind" and "alias" to grab the rest of the line, not just arg(3)
* don't clear impulse events every frame, so a weapon change will wait
for attack completion
* added "wait" command. These changes allow:
bind g "impulse 5 ; +attack ; wait ; -attack ; impulse 2"
* got demos mostly working
+ worry about shutting down listen servers before connect / demo playback
+ remove item precache
+ finish switching lights
+ keyboard freelook
+ do clients need keepalives?
spikes coming through walls
explosive nails
change thresholding on trimapper in higher res!!!!!
muzzle flashes
-----------------------------------------
John Carmack's .plan for Mar 23, 1996
-----------------------------------------
* lots of cleanup in cl_ code
* killed looping sounds at disconnect
* faded palette back down at disconnect
* fixed bobj miscount and added bobj count error check
* interpolated angles and velocities for low update connections
* finished demo code
+ sound caching....
+ looping sound still doesn't work on my system
+ can't throw out spawned sounds too far away, because of teleporters
+ ammo box on sbar doesn't allways update after death
+ two state door
+ allocate translations on the hunk to save single player memory
remove headless / legless death frames
swimming frames
server viewpos problem
relinking by size still isn't right
explosion core
precache_auxsound for non-essential
don't cache player model in single player
-----------------------------------------
John Carmack's .plan for Mar 24, 1996
-----------------------------------------
* level to level transitions
* finished light modifications for switchables
* fixed bug that caused some entities to not appear
+ weird field trashing bug
+ don't clear sound precache on non-progrestart
+ stuff not showing up in jccode
+ savegames
+ nail angles are wrong
+ cought on buttons in johnc99
+ is anything confused by sv.active on clients during a net game?
stereo positioning is wrong
nograv
handle coop games
make lightning a tent
worry about security risks with pathnames for demos and savegames
need to save lightstyles in savegames
non-convex bsp output
-----------------------------------------
John Carmack's .plan for Mar 27, 1996
-----------------------------------------
* fixed soundlist
* explosion through walls doesn't damage
* monsters don't melee through walls
* fixed memory trash from restarts
* changed signon messages to more client requests, removed buffers
* invisible clip solids
+ wake up monsters through walls
+ is vid palette setting fixed?
+ remove up/down movement from non-noclip
+ entity bsp clipping for shots and monster moves
+ make QuakeEd center on start position
+ climb out of water
+ light switches?
bsp sticking point
no aim key
don't include trigger and clip mip textures in files?
-----------------------------------------
John Carmack's .plan for Mar 28, 1996
-----------------------------------------
* fixed efrag allocation on restart (really)
* fixed static sprite placement
* fixed light casting from torches
* fixed spawning angles
* fixed savegame and multi player baseline object startup
* made QuakeEd go to info_playerstart at level load
+ fix damage trace through entities
+ keyboard freelook
spinning gibs on floor
very high framerates mess up
extra back attack damage
buffered IO problem on romero's machine
stereo still isn't right
partical puffs
-----------------------------------------
John Carmack's .plan for Mar 30, 1996
-----------------------------------------
+ dynamic lights....
+ grenade activation size
+ entnum optimization
+ make progs use pointers for edicts?
+ sideways spikes
esc = floating menus, with console option
remove avelocity?
track down the _SCR_UpdateScreen <cycle 2> [23] from profile
clean up search path syntax
fix style lighting on alias models
more #ifdef PARANOID
-----------------------------------------
John Carmack's .plan for Mar 31, 1996
-----------------------------------------
* changed edicts from indexes to pointers
* fixed byte order problem in qcc
* fixed backward movement dies after many savegames
* fixed bus error in edict linking after a model remove
* cleared rotate flag on CL_SetModelFlags
* made items move with pushers
+ animating textures
+ vis problems out of water in jrbase2
+ get trail characteristics from model name? (now a model flag)
+ switched lights don't work right after restart
+ aiming at fast moving entities
+ center view key
light filter in quakeed is broken
per-entity ambient light value
don't play door trigger sounds if full up
shoot over fallen zombies
no changeup to grenades
add targets to doors
auto pause when console
player gib packages
null bobjmodel in singleplayer
wizard bombs through walls
drowning time?
shambler lightning
hang when starting a mistyped level (time based?)
old deathmatch rules?
lockup overnight
don't hook keyboard on dos dedicated server
cycle edict allocation so spikes aren't reused frequently
make spikes rotate
-----------------------------------------
John Carmack's .plan for Apr 01, 1996
-----------------------------------------
* added $flags to models for client side stuff (rotations, trails)
* allowed coop play
* started using Host_Error for client errors
* lint
+ animating textures
+ NULL connect
+ connect while in a game crashes
different sfx cache size for client only?
frame chnaging while paused?
item targets
respawn in coop
-----------------------------------------
John Carmack's .plan for Apr 02, 1996
-----------------------------------------
* exploded wizard bombs on death
* server optimization
* fixed angle on loadgame and coop
* average monster velocity for aiming
* made wizard fall properly
+ running into nails
+ can still get shot out of the world
+ dynamically allocate color translation tables
+ pause work
+ backpack dropping
water stair climbing
check door linking , regarding the partial closing
restart level spawn parms
dead player save que
quakeed magnified texture view
ambient + gravity attributes
load fames
restart without killing server
no-aim option
-----------------------------------------
John Carmack's .plan for Apr 03, 1996
-----------------------------------------
* made func_wall and removed the dm_only / not_dm
+ sky fuckups
+ single pixel movement quantization?
+ falling out of world
auto aiming sucks
remove superscaling
s->relasecount error
.cfg default for exec
frame too fast bug causing lockups
last request/reply
registration trademark!!!
explosion cores
check side to side movement for big attacks like the shamblers?
change backpack content messages
combine model and sound cache!
better PVS with more leafs?
-----------------------------------------
John Carmack's .plan for Apr 04, 1996
-----------------------------------------
+ teleport flash is broken
+ order of monster fall to ground
multiple input drivers (hard coded)
save cfg
bounce head gib?
wall scaling is step up?
-----------------------------------------
John Carmack's .plan for Apr 05, 1996
-----------------------------------------
+ sound loop points
multi life / level demos
precache_sound return string value
make jump buffered edge triggered
clipping hull lips
hang after sitting overnight on dos
-----------------------------------------
John Carmack's .plan for Apr 06, 1996
-----------------------------------------
use PVS to speed line of sight checks? Is that any faster than doing
the line check, because it involves two point checks?
swooping ai movement by checking distance to impact with forward trace?
make minimum frame time a cvar, so you could clamp to 10fps
runmove() builtin for progs?
draw triggers debugging option (z buffer it)?
look towards killer when dead
-----------------------------------------
John Carmack's .plan for Apr 07, 1996
-----------------------------------------
* fixed tfog
* fixed removal of entities over net
-----------------------------------------
John Carmack's .plan for Apr 08, 1996
-----------------------------------------
* fixed stray rocket trails
* fixed coordinate integralization problem
* identified the Aim() time sink
* fixed smooth step up
* got new clipping code running
+ sound looping prob is due to decimation
+ expand numtextures to a word from a byte
+ remove FL_ONGROUND after being pushed
+ still damage through walls
+ counting trigger
+ paused graphic
+ demon freeze jump
add s and t vectors to the plane description in .map files
remove the pad around bmodels bounds
make pushmove area check instead of linear search
server pass full pathnames to client
dos crashes overnight (timing wraparound bug?)
quake logo screen needs filtering
shoot all the way through walls with objects
wizard fireball
plat problem in tim17
shambler lightning
weird grenade launch spot
auto pause on console
hold "other" on delay triggers
gib stuff
-----------------------------------------
John Carmack's .plan for Apr 09, 1996
-----------------------------------------
* fixed crash when -NXHost on quake
* added +precache 0 option to speed up WAN loading
* fixed sound loading bug
* added loop lengths to sound loading
* increased MAX_MAP_ENTSTING
+ quakeEd filter brushes
+ 16 bit looping sounds hang the dos game
QuakeEd showed "Helvetica-Bold" in entity comment field. Scary.
maake treading water better
trigger to turn off spikeshooters
save the entity string memory
blocked bmodels are conveyors?
-----------------------------------------
John Carmack's .plan for Apr 10, 1996
-----------------------------------------
* made no-light levels auto fullbright
* finished second clipping hull
+ zombies on the ground can't be gibbed
+ listen servers not working?
error check all strcpy in QuakeEd
slow slide clipping with ogre hull?
grenades should trigger shootable triggers / objects
looping sound that didn't stop
-----------------------------------------
John Carmack's .plan for Apr 11, 1996
-----------------------------------------
* fixed damage through walls
* fixed gibbing downed zombies
+ KEY_SEMICOLON
pointing?
ten second max looping sound?
integral sound stepping?
the damage byte stream is what is causing packet overflows
debounce jump
is there a mouse call to find the number of buttons?
fix palette after vid change...
too many vid_palette calls
DON'T FLUSH THE ENTIRE DAMN SCREEN!
fix snd_next to be snd_none
change sys_loadfile to com_loadfile
"system is little endien" message as optional verbose debug
sometimes shotgun sound doesn't play
double doors independantly closing
server speed went back up
fix auto aiming
no gun aiming
no change to grenade launcher
ogre grenades
zombie gibs
r_ambient
fixed angle for grenades when stopped
client side shambler lightning
gravity on corpses pushed into blocking positions
slime darkness value
sky time problem
integrate dirpack with qcc
check precedence of test directories and pak files
rocket auto aiming is not working right
amtele2 teleport problem
no save when dead
memory fail to console, not dos
still overrun nails
-----------------------------------------
John Carmack's .plan for Apr 12, 1996
-----------------------------------------
<at raven>
-----------------------------------------
John Carmack's .plan for Apr 13, 1996
-----------------------------------------
<at raven>
-----------------------------------------
John Carmack's .plan for Apr 14, 1996
-----------------------------------------
* fix 8 bit palette in NS
* modified modelgen to allow multiple models in one file
* fixed qbsp to detect cases of no entities placed and skip fill outside
* new wizard spike attacks
fix corrected palette at mode change
make dos utility tree
make QuakeEd's menu commands variable length
debounce console
debounce jump
remote slist
slist with user names
fall out of world when dead
s_releasesound: s->refcount < 0
sound cache parameters
clear pause on restart
check pak file overriding
-----------------------------------------
John Carmack's .plan for Apr 15, 1996
-----------------------------------------
* lowered slime cshift
* wizard slide move
* dog frames
* fixed auto aiming when dead on
* soldier less hp + tuning
* various gibs
* dynamically allocated scores structure
* trigger_counter
* door TOGGLE
* fixed forking bug that trashed pointfiles
* fixed sky fuckup
+ fix demon pop jump
+ zombie get up inside other
+ zombie gib throwing
+ lose onfloor when pushed
+ animating textures
+ schurch weirdness
make light styles global prog vars
base4 entrance door sucks
back attack damage
fix dynamic light values of players
aim up at nearby monsters
load pcxs in lbmlib
text triggers
swim slow straight up?
temp entities
client sound calls
zombie heads bouncing
bouncing monsters out of the world
gib bodies between pushers
-----------------------------------------
John Carmack's .plan for Apr 16, 1996
-----------------------------------------
* reduced bodies to points on squash
* turned off onground when pushed
* new demon jumping
* fixed zombie stand ups
* fixed zombie standing off floor
* fixed uninitialized vector in movestep
* fixed scr_ofs cheat
* fixed upjump cheat
* _field names are ignored by quake
* fixed bottom checking
* fixed up fov
* moved icons into view screen
* added pause icon and showpause cvar
* renamed SOLID_EDGE to SOLID_TRIGGER
* delayed monster drop to floor
+ mbug.map texture toastage
+ save / loadgame start view angle
+ fix demon popjump
+ flickering ammo box on restart
+ weapons not following to next level?
+ knights not coming off paths when angered
+ load / save games are broken
+ pause veloicty spin
+ reset lightstyles on level load
more screen turbulent warps
default fields in /*QuakeEd header
is draworder working in deathmatch?
enforcer no melee attack, side step
triangle transition size
open secret doors with area effects
alias model racing stripe
shoot over grounded zombies
activatable trigger
wizard altitude changes
enforcers need to drop ammo
player torso gib shouldn't be used for most monsters
teleportation issues
some keyboard keys make it in while changing levels
make a base directory variable for all util scripts?
invisibility (remove weapon)
flyjumping up slopes
better polygon lighting
track killer when dead
allocate cl_entities on the hunk
start dead zombies
cvar changing buttons
invulnerability powerup
raise / lower weapons
is qc pausetime needed?
delay item drop to floor?
run first second of game before entering player
-----------------------------------------
John Carmack's .plan for Apr 18, 1996
-----------------------------------------
* finished new bsp portalization process: 20% faster, 1/5 the memory
* fixed monster repeat teleport
* klook and mlook
* fixed monster stepping up into mid air
* what breaks switchable lights is a bsp->entities without running light
* changed network startup to use signon ready flags
* added extra face-on wall friction
+ fix ammo between levels
+ dark knight
+ message wasn't a nop
+ schurch zombie graves
+ sandy's bug1
+ slow drifting spikes
+ kill wizard spikes after their death
+ s_refcount bug still exists
+ wizard spike starters spawn inside walls
+ plat dmg fields
+ longer pause for ogre first shot
should demons step up before leaping?
shooting spikes out of rockets
need mouse configuration in the menus
try removing bounding pad on bmodels
add fadable light switches
ability to bind specifically to key up / down ?
bound ammo on weapon pickup
lava damage for monsters
error check add to reliable messages
less accurate ogre tossing
-----------------------------------------
John Carmack's .plan for Apr 19, 1996
-----------------------------------------
* fixed world/entity clear allsolid flag bug
* tracked down problem with zombies popping out of graves
* fixed path follow when in combat mode bug
* fixed standing on top of monster stuck bug
* animating textures
+ don't report surface cache size when caching sounds
+ pick up items multiple times when on dropping plat
+ pause continues across restarts, but plaque doesn't
+ overnight lockups
+ debounce jump
+ wizard spikes start in walls
overbright light palettes?
slope auto pitch needs work
secret area counters
don't slide on slopes
zombies need to be gibed by pushers
monsters trigger plats
trigger_notouch
warnings on coplanar brushes in bsp?
only disable auto center when head is under water, not feet
debounce escape
lightstyles in savegames
sticky bsp corners
wizard partical centers
make triggers use BSP models?
better sky management
-----------------------------------------
John Carmack's .plan for Apr 20, 1996
-----------------------------------------
* added hell knight
* Hunk_ReportSurfaceCache() instead of automatic
* fixed multiple touch on items riding plats
* helped non-solid and trigger items push better
* cleared pause on map restart
* fixed level change parms
* dynamic lights!
* special wide box for missiles against monsters
+ larger rocket radius
+ slow motion screwups
+ nailgun stops rotating when riding a plat
+ stuck roll angle
use knight kneeling frames
zombie crucified frames
looping sound while invisible
make running direction changes smoother
error check all file opens?
load temp file on stack if possible?
remove SOLID_SLIDEBOX
put auto aiming back in?
move rule checking to end of frame
quicksave / quickload
dissallow spawn parms on deathmatch
make connect to server send version number
make temp entities auto remove
no body que?
water up swim speed
tim was getting stuck in deathmatch
-----------------------------------------
John Carmack's .plan for Apr 21, 1996
-----------------------------------------
* fixed texture animation cycling roundoff bug
* fixed up missile clipping
* fixed super spike -1 ammo and use 3 ammo bugs
* checked deathmatch before precache on monsters
* jump out of water
* debounced jump
+ check extranious messages on connect
+ continuous looping sounds
+ button texture changing
+ state textures for buttons?
+ remove the legless / headless player death frames
why is shambler lightning funny colors under NS?
extra point radius shoots through walls
big sound cache for deathmatch
abort load command?
is max_health ever used in progs?
don't go to frag scoreboard when single player dies
clipping against surfaces causes multiple slime damages in multiplayer
only?
treading water less bouncy
jetpack model is saved wrong
super size boss
-----------------------------------------
John Carmack's .plan for Apr 23, 1996
-----------------------------------------
* changed network update to client PVS specific
* fixed broadcast svc_setangle on player connect
* fixed "Message was extranious" connection bug
* partical fields
* invisibility model switching
* effects passing from server
* invincibility
+ self.modified is no longer needed with PVS updates
+ empty space back fill Z buffer bug
vertical aiming sometimes shoots at a farther object
over long latency lines, should shots come from the position the player
was
viewing, or the true position?
combine client->active and client->spawned into a client->state
backpack message should include weapon gainsed
throw monsters around from explosions
drop wizard by half gravity
make tri subdividing rasterizer based on screen size! (huge models go
slow)
american got a prog to crash
allow listen servers to have a sys_ticrate?
cut back on use of global host_client
new teleport surface warp
body que is broken
-----------------------------------------
John Carmack's .plan for Apr 24, 1996
-----------------------------------------
* muzzle flashes
* eat keys between levels
* added new pain frames for soldiers
* fix savegames with PVS update
* new status bar updates
* fixed packet overflow and noclip with PVS update
* synced zombie throws from exact position
* put in new ogre grenade
* no span lighting in subdivided triangle code
* better gibbing code
* vid_size for nextstep
* determined the hang is in the sound mixing loop never exiting
+ auto pause with console
+ more time for ogres before launching grenades
+ is restart doing a progrestart?
+ fix gib head bbox
+ head outside level
+ backpacks don't drop when floor is pulled out
+ movetype_missiles need to be intangible against owners
+ gib dogs
+ teleport flashes aren't in the pvs
+ soldier3 new pain frames
+ fix high framerate bug
minimum ambient light field for bmodels
make it easier to jump out of water
default water treading
better discrimination of slime and water
need to cycle allocations of temporary entities to prevent false reuse
repeating bounce sound on dropping plats
crush monsters extremely by plats
about to drown warning noise
load last savegame by default
monsters burn in lava
float particals from tport pads
don't switch to frag bar on single player death
slope lookup
got stuck in a downed zombie
make an error message in qcc for function calls inside a parm list
polygon torches
need to make impulses reliable, or looping sounds may not be killed
restart game with entrance spawn parms
allways test demos and savegames after a net change!
option to match ping times for fair deathmatches?
get bonus items on top of doors spawning deterministically
selective cheat for testing
-----------------------------------------
John Carmack's .plan for Apr 25, 1996
-----------------------------------------
* increased spread on super shotgun
* rewrote sound loading and mixing
* fixed sound refcounting bug
* fixed sound not playing bug
* fixed sound hanging bug
+ axe
colored lighting sample
lightning gun
light bmodels
don't let the view model go full dark
-----------------------------------------
John Carmack's .plan for Apr 26, 1996
-----------------------------------------
* bringing down console auto pauses in single player
* fixed ultra-slow motion console movement bug on bad bat file startup
* fixed slow motion when > 72 hz framerate
* added axe
* never change up to grenade or rocket launcher
* added lower sample rate and bit sound loading
* stored the fullbright index out in the colormap lump
* new enforcer pain frames / code
* added alternate animation sequences for textures
+ still running into spikes
+ fix host_framerate
+ drop backpacks when floor removed
bound ammo maxs on weapon pickup as well as ammo
diminish sounds through walls
partical puff and light flash from spike shooters
do a disk icon when caching
proper alias model bbox efrags
allow monster hearing at farther distances
animating texture speed control
slipgate lightning sprite
winking out door in tim6
pusher problem on smotte (climbing out of water on a bmodel)
need to allocate as large a sound cache as possible
wizards die on top of spikes in mid air?
doors shouldn't start move sound if blocked open
don't slide down slopes
cache alias models
-----------------------------------------
John Carmack's .plan for Apr 27, 1996
-----------------------------------------
* alternate texture animation sequences for depressed buttons
* dropped knight and ogre melee damage
* arbitrary texture axis on surfaces through all the utilities
+ remove .modified
+ need to make qbsp do 256 subdivision based on texture axis
+ monsters still hit pathcorners when hostile
+ demon jumps don't do damage
triggering an open door still makes a noise
allow turning off styled lights
QuakeEd texture x size field is clamping at 64 min
faster mouse forward/backward?
is it jumping higher when running?
gun puffs still hit sky
get all visible entities into the low 256 or 128 edicts
force low quality sounds on low memory machines
still have some light edge blobs
-----------------------------------------
John Carmack's .plan for Apr 29, 1996
-----------------------------------------
* fixed light sample determination for all rotations and scales
* pulled wizard shot starts inside bounding box
* fixed demon jump damage
* objects never clip against their owner, and vice versa
* ogre pain frames
* fix pathcorner bug
* don't let monsters see into water
* don't allow dead monsters to be used by triggers
* reset player's gib head to proper bounding box
* gib dogs
* killed avelocity when bounce objects stop
* toss objects drop with plats now
* remove progrestart
* wizard spikes don't fire if dead
* make wizard drop altitude
* better stereo seperation
+ check message triggers
alpha qlumpy still has wrong type numbers
help screen?
auto reload savegame
add pak building to qcc
let doors target other items
guys gibbed by boxes go to no such frame?
is axe view skipping frames?
do player pain frames cause weapon frame skipping?
highlight current weapon in status bar
dynamically light bmodels
swimming motion
explosions need to reveal secrets
-----------------------------------------
John Carmack's .plan for Apr 30, 1996
-----------------------------------------
* clamp health and ammo on level changes
* removed modified field from edicts
* added pause before ogre first attack
* counter triggers don't fire on -1 or less
* added "give" command for testing
* added highlight on active weapon on status bar
* fixed message triggers
* styled light now correctly light alias models
* restart level with entering items
* increased rocket radius damage
* new swimming code
* changed sizedown aspect ration to mimic full screen instead of status
bar
fix progrestart
sound streaming
klook is broken
don't slide on slopes
optimize server code more
get fakeplayer spots working again as monster targets
longer pause for ogre first shot
knights run up ramp on tim7 and stand in air
style light alias models
ogre only fall down when surprise pain
monsters should get angry at each other more often
increment sound channels so items never collide
powerups
vomitus
hell knight
blobs
work on dogs
monsters sim up/down
jump triggers
low grav aiming ogres
throw monsters with explosions
per level-gravity setting
explosion cores
monsters take damage in lava
turn towards killer when dead
slope lookup needs to match stairs
no monster pain frames in ultra-hard mode
monster moves after killing player
disk icon during savegame
smooth center instead of snap center
sometimes multiple death sounds don't start
don't use stdio lib from savegame
"player switches to " messages
info bar in single player: current secrets, kills, level name, etc
load game unpauses
pick up own grenades without explosions?
monsters fight each other after player death
allways switch to picked up weapons
switch to super weapons automatically?
light styles in savegame
prog access uninitialized edict field causes bus error
get 64 bit clean (progs overlay pointers and floats)
allow multiple pack files
-----------------------------------------
John Carmack's .plan for May 01, 1996
-----------------------------------------
* background noise track code
* fixed swimming nan bug
* fixed Sandy's quakeed crahser
* QuakeEd: turn region off when going to new map
* automatic sound decimation based on available memory
* explosion sprite cores
* QuakeEd: don't go to worldspawn
* qcc is now fully recursive
* buttons only pressable by player
+ the explosion cores are still MOVETYPE_TOSS, not MOVETYPE_NONE
+ set gravity on secret level
+ host_framerate cvar is broken
+ shootable buttons
+ wizards on paths go to wrong heights
+ make knights not move when in melee range
+ light styles in savegames
+ QuakeEd: filter world
+ QuakeEd: remove check for filtered brush
fish fly out of water
make door triggers not box sized
check respawning on tim5 armor that drops down
make an end of frame call for each player
allow sprites to have a faked forward Z value? allways?
key alias models
-----------------------------------------
John Carmack's .plan for May 02, 1996
-----------------------------------------
* changed grenade cores to MOVETYPE_NONE
* set gravity in worldspawn
* fixed host_framerate
* stopped sounds before server shutdown
* clear all controls on first client frame
* view.map, viewmodel, viewframe, viewprev, viewnaext commands
* dropped ogre backpack to 2 grenades
* fail soft on video mode memory allocation
* checked console resize in vid mode set, so size messages never get lost
* shootable buttons
+ edge light seams
+ increase max packet size during development?
+ rotate dogs slower?
+ monsters drop to floor after teleport
+ need very obvious sound for sequence completion
+ model viewer inside quake
+ cache alias models, if only to help level to level change speeds
+ cache heap compaction
+ remove bsp->leaktest from QuakeEd
+ dog leaping
+ sight sound on monster use
+ allways switch to picked up weapons
+ blink weapon icons on status bar when picked up
+ climb out of water bug
+ water warp option in all vid modes
+ make qcc 64 bit clean by using doubles instead of floats on 64 bit
archs?
intermission levels
spritegen has funny pathname determination
demo fast forward
seperate impact from damage
combine some of the state flags in both client_t and server_t
ambient light option on bmodels
palette should be reset whenever console goes full creen
should all +- binding have the keyname prepended to avoid the multiple
downs?
seperate mouse scaling for turning and moving
make a "soundshot" option fortesting soundcode in ansi spec makefile?
raise / lower weapons
change weapon after firing last round
flying fish
change entity flags to effects
should host_error try to cleanup aborted servers and clients?
how many hunk allocs are there? save memory by combining?
QuakeEd: filter based on skill levels
check trigger firing when spawned inside one
remove input line completely when not accepting input
data checksums for remote servers
shrink net packet sizes
-----------------------------------------
John Carmack's .plan for May 03, 1996
-----------------------------------------
* reduced rocket refire rate
* removed ogre wait after awake
* added shambler pain finished time
* new caching code
+ staircase rail on tim6 needs to be taller
+ normal server disconnect is reported as a host_error
+ jrbase1 up arrow out of slime is near a step
+ bad changeup to green armor from yellow? level transition?
+ are demo playback speeds correct?
+ stair walking underwater
+ ogre explosions need cores
+ teleporting monsters left a statues
+ shootable buttons don't anger monters properly
+ cycle weapon command
+ pain channel is same as weapon channel
- cut swimming speed some
- go through more get up frames on zombie before failing
+ got stuck in a dropping zombie
+ wizads on paths drop to ground
+ torch models
+ remove keys on use
- different keys on status bar for different levels
+ all items in inventory need to blink on pickup
+ soldiers seeing into water on tim6?
savegames cause enemy items...
different color console fonts
monsters need to hear from farther away
think about patch / update plan
precache_ambient
alternate surface warp
killing ogre while on falling platform left him in the air
better treading water
better search path handling
make a progs / models test directory for kevin
blink key icons on doors
default.cfg allways goes to knwon state
animate key icon on doors when failing to open
-----------------------------------------
John Carmack's .plan for May 04, 1996
-----------------------------------------
* allow "quake f:\quake\id1" for better ISDN development
* dynamic caching of alias models
* fix the surface cache size instead of letting it fluctuate
* explosion cores for boxes and ogre grenades
* transfer monster anger from button press
* fix trigger message printing for secret level
* added notouch flag for triggers
* added nomessage flag for counter triggers
* fixed secret counter triggers that are fired instead of touched
* fixed fired button activator handling
* fixed wizard path following
+ alias model torches...
+ don't spawn triggers for key doors, just use touch
+ make sound channel 0 never match
+ jrbase1: don't put guard inside ammo box, superhealth at end is no good,
bridge delay is wrong, can still fall into slime
+ debug the stdio library problems!
tim5 armor drop on ogre fucks up
remove UPDATE_BINARY and pushmove
make +_ key commands pass source as second parameter
cace console background?
more alert monsters?
demo cycling breaks netgames
automatic treading water
squish to bloody paste on interference
-----------------------------------------
John Carmack's .plan for May 05, 1996
-----------------------------------------
* removed player entity from own entity set
* fixed memory trash bug from not flushing surface cache before
changelevel
* use all memory between the hunks as dynamic cache
+ dump time between levels
+ sticky bsp corner
+ fix wall jump scaling
+ better backpack messages
QuakeEd is doing something over the net on NXPings
bloody face health indicator?
crosshair?
don't pass angles, frame, etc on viewentity
rockets spawn past point blank
Single player names on qsockets are backwards
make dead player still solid to monsters to prevent seeing inside models?
-----------------------------------------
John Carmack's .plan for May 06, 1996
-----------------------------------------
* removed MOUSE1, MOUSE2, MOUSE3, added K_MOUSE1, etc for binding
* check for spurious mouse buttons
* fixed stair climbing underwater
* trigger_onlyregistered
* fixed door sound loopinhg problems
* button configuration
* made monsters stand still at point blank range
* idle view motion
* monsters get mad at each other on any damage
+ cache movement on hunk allocs
+ hi res modes with new code are broken
+ dead guy still firing / pushed into floor
+ fake pause causes problems when map from console
+ zombie shouldn't go into pain frame when down
+ change weapon command
+ is DMA left enambled in sound code?
respawning of items that dropped down with a plat
rocket points block plats
don't clear text on vid mode change
grenade bounced off ogre
bind # option for weird controllers
key sticking
investigate the vis problems
make monsters head for their spawn points after they kill the player
camera follow killer
destroy backpacks when doors close on them
lightning gun model
lightning gun no damage to shambler
does shambler bolt hurt monsters?
-----------------------------------------
John Carmack's .plan for May 07, 1996
-----------------------------------------
* fixed point / point missile interactions
* don't change weapons on pickup in deathmatch
* made vis 30% faster
* fixed falling in floor
* fixed and armor plated the jump out of water code
* fixed jump monkey climbing
* dissalow "kill" when allready dead
* all hell knight actions
* COM_LoadStackFile
* cache movement
* fixed shambler pain frame
+ stair climbing clipping against walls is messed up a bit
+ monsters need to drop to floor after teleport
+ menu01
+ loading on stack causes errors on dos
+ flashing disk icon
+ make caching perfect
+ monsters not fight each other if not seen player yet
+ make lighting gun consume cells
+ falling into floor bug
+ zombie clipping
+ cycling channel allocation
+ no Host_Error: for valid disconnect
+ body que isn't working
shooting rockets at point blank
deathmatch switch to better weapon again
lightning needs to pitch
control animating texture speed
need to cut the update message size
sfx volume option
player only checkbox for doors and triggers
launch lightning from exact spots
finalize HMD interface
repeating grenade bounce on some slopes
jump out of water automatically when swim up to an edge, no need to jump
new ground explosions
auto pitch went into look down for no reason
gib velocity reletive to damage?
-----------------------------------------
John Carmack's .plan for May 08, 1996
-----------------------------------------
* fixed crash on machines with no sound card
* made texmake grab new palette
* removed tent generated soudns from progs
* fixed enemy anger on damage
* sound channel 0 never overrides another sound
* basic blob
* light styles in savegames
* Cash's fix for savegame slowness
* door messages
* fixed door retriggering sound and mixed size letdowns
* changed zombie pain / clipping code
* enforcer backpacks and alt death frames
* trigger monsters play awake sound
* longer ogre aim time
* fixed checkattacks crossing water
* new bit coding on entity updates
+ lightstyles in savegames
+ broke multiple message triggers
+ key door sounds
+ messages on double doors
+ messages on secret doors
+ 3 digit frag bar
+ light edges
+ change weapon command
+ center print messages
+ hell knight slice
+ more wizard death frames
+ underwater noise
make gib entity rotation client side only
ambient level on doors
decouple weapon animations from pain, etc
light bmodels
point blank rockets
look at killer monster
swing weapons down
turn view when axe fighting
grenade destroy bodies
lightning/rocket shooter
swimming frames
teleport warping
teleporter surface warp
blow up corpses
get all visible enttites into low 128 or 256
-----------------------------------------
John Carmack's .plan for May 09, 1996
-----------------------------------------
* fixed door linking / messages / targeting
* remove onground after teleport
* fixed zombie pain and getting up inside players
* hell knights don't anger each other
* added source entity to prog traceline
* fixed point blank rocket bug
* new light sample point determination
+ no host_error on proper disconnect
+ change weapon
+ powerups
+ cell packs
+ fix pausing on menu / console
+ make key doors touch activated instead of triggered
+ lightning gun use ammo
+ word wrap consoke
+ monster jump triggers
- blob spit attack
is turning not happening when in contact with monsters?
client side lightning
lightning gun
enforcer bolt
more pain feedback
setting up the vid mode then changing levels can cause out of memory
give a spike when hit by spikeshooter
monsters burning in slime
hknight throw a spike occasionally when charging
demon jump down from higher
higher demon jumps
enforcers still see underwater
-----------------------------------------
John Carmack's .plan for May 10, 1996
-----------------------------------------
* fixed undead demon bug
* new main menu
* everything splashes on entering water
* bounded ammo on weapon pickup
* added cell models and stats
* Cash's screenshot error check
* Cash's Host_EndGame
* Cash's net packetsize bump
* disc icon during IO
* monster jump triggers
* cache optimizations in edge refresh
+ cycle new allocation numbers
+ make qcc do pack files to a destination
+ dog leaping
+ wizard death frames
+ death popping
+ step up wall clipping
+ crucified zombie
+ nudge weapons with sized down screens
+ 64 bit clean qcc
remove drop to floor?
make Quit go back to demos and reset controls for E3
lots of demos
slime damage for all monsters
pass PVS list to refresh instead of passing efrags
remap stupid entities (item_weapon)
sound for all monsters hitting ground
reset the bonus items in monsters
refix the rocket stuff after demo
-----------------------------------------
John Carmack's .plan for May 11, 1996
-----------------------------------------
* wizard death frames
* dog leaping
* make key doors touches
* increased air time by two seconds
* new keyboard pitch aiming aids
* crucified zombies
* cemter printing
+ tally monster deaths at end instead of counting at kills (monster on
monster)
+ parse \n out of messages
+ status bar feedback
+ klook
weapon dropping
weapon jumping
mess with configuration screen
move all messages into a language qc file
fix frag bar
some side velocity on death
check monster full turn before firing
check exit hang on beta machine
puff+light for spike shooters
tim5 armor fall on ogre bug
view track killer
enforcer bolts
less monster alertness when high or low
blob spawn ceiling
fix coop
-----------------------------------------
John Carmack's .plan for May 12, 1996
-----------------------------------------
* catch overbackoff in movement sliding code
* extended key/value size in utils
* added flame alias model
* \n in epair text
* fixed slope by wall false step in movement code
* hacked around sticky corner problem
* weapon/item flashing on inventory bar
* fixed soundinfo exploded on a no sound card system
* optimized snd paint and transfer
+ change weapon
+ jrstart: no fall in water by medium
+ demo loops
+ player stats on frag bar
+ nail gun alternating
+ cycle edict allocations
+ jrstart: light up black corners
+ clear center print when console drops
axe view tilt
better underwater bubble control
track killer
kick monsters off edges in pain/death
jump landing frames
new explosions
-----------------------------------------
John Carmack's .plan for May 13, 1996
-----------------------------------------
* weapon cuycling command
* trapped bug with maps without textures
* increase key/value size in QuakeEd
* qcc -pak option
* "flechettes" to nails
* added "killtarget" on all SUB_UseTargets
* secret doors do SUB_UseTargets
* secret door touch messages
* cycle entity allocation
* versioning in savegames
* save skill in savegame
* don't set onground when standing on another monster
* fixed loadgame during demo bug
* put falling crunch on voice channel so it overrides pain sound
* fopen demos inside packfile
* automatic ambient sounds
+ bad recursive explobox?
+ ambient sounder tents
+ tab stats
+ center print sequence stuff
+ demo files in packfiles
+ powerups
+ underwater looping sound
+ pass source as parm for +defs
coop start spot
don't accept mouse view when paused
"you got # shells" for item messags
precache_ambient
visbug.map
seperate impact from damage
create pak files with search path?
tim5 drop item on monster bug
texture animation speeds
squash to paste
doubled "clearing memory"
parse /* */ comments in config files
explosion has bad colors in it
proof of purchase
lightning gun
treading water
looking down towards ground bug
deathmatch level transitions
better lava balls
is zbuffer precision maximized?
alias subdivider used wrongly in zoomin mode
permanently shifted view angle on restart
optimized single channel ambient mixing
are 0 volume sounds hanging around?
jrbase3 old bsp explosion
auto jump out of water
demo file version numbers
-----------------------------------------
John Carmack's .plan for May 14, 1996
-----------------------------------------
* fixed recursive barrel death
* new characters
* better ambient name parsing
* con_dpritnf for developer messages
* start messages out of maps
* directional touch triggers
* remove qtest license stuff
* fixed off by one error with ambient leaf calc
* level spawning messages
* clear input line every time console goes up
* moved all non-refresh vid.buffer access into draw.c
* removed alphalumpy -- use qlumpy
* automatic extension and dest file creation for qbsp
* merged id1 and id1_ development hierarchies
* Made QuakeEd projects directory independent
+ raven's bug map
+ static point ambients
+ hell knight smash
+ flameb1 - 11
+ explodes on manually going to bad map
+ play humm1 for teleport textures
+ parms for all + commands to identify source
blood/chunk spray direction from claw attacks
shootable base texture needs an off frame
lint the utilities
cache console background
make all jump trigger targets have a land sound
center menu in high res
slipgate animations
loading plaque, no console between levels
supress input line
fix timedemo with cycling demos
gun jerking
!cl.worldmodel is no longer valid?
mike signon messages stay in notify lines
break off anger at another monster if can't reach in ten seconds
debounce all keys?
center print level messages
still dropping pack in single player?
debug alpha modelgen
-----------------------------------------
John Carmack's .plan for May 15, 1996
-----------------------------------------
* fixed bug with cl.disconnect not clearing worldmodel
* linted and asnified qbsp/light/vis
* changed sbar update model
* powerup colors and icons
* fixed console ] gets messed up with escape/escape
* fixed weapon icon flash going from level to level
* solo console
* blob fly1-4
+ QuakeEd: don't allow retexture in wireframe mode
+ QuakeEd: clear wad on bad load
+ unfork qbsp
+ body que
remove any remaining static limits in qbsp
split client state into level - game - invocation
eyes on status bar for invisibility
accumulate damages so quad dmg shotgun can gib
fix spit.map
optional FOV cull for modem?
super armor number
server send prog checksum to clients
fix hard coded location of status bar numbers in wad
Cash's teleporter issue
fake some angles to hide latency on movement
faster mouse forward/backward
+ QuakeEd: clear brush heights on new level
more edicts
+strafelook
toughen up shambler
no demon/demon blob/blob violence
hell knight anims
dopg death frames
outside level plays all ambient sounds
cl.active isn't good anymore
demo loops crew multiplayer
give for powerups
make clip brush only models work
fix bug_numpoints_on_hp.map
-----------------------------------------
John Carmack's .plan for May 18, 1996
-----------------------------------------
* make func_wall toggle frame on use
* QuakeEd: remove bsp->leaktest
* QuakeEd: filter world
* QuakeEd: remove clicked on regioned brush message
* QuakeEd: don't allow retexture in wireframe mode
* QuakeEd: new brush heights are now snapped to grid
* default .bsp extension to bspinfo
+ enforcer bolts
+ sound for all monsters hitting ground
+ 64 bit clean qcc
+ parms for all + commands to identify source
+ body que isn't working
teleport effect
global sfx volume option
frag bar highlight
permanently shifted view angle on restart
goddamn dos \r text shit in output from NT tools messes up NeXT tools
check for small area polygons and portals
-----------------------------------------
John Carmack's .plan for May 20, 1996
-----------------------------------------
* server sounds parameter specifies soundtrack
* fixed cursor bug when clearing input line on console raise
* fixed problem with e4 ambient (32 bit overflow)
* ambient teleporter sound
* snd_show debugging tool
* fixed underwater grenade sounds
* fixed shambler hold attack until in range
* ogre chainsaw on side swing
* claw meat spray
* f1 toggle help
* no end of level stats on empty levels
* fixed tilted head savegame bug
* don't drop backpack in single player
* wrote unpack utility
+ teleport static sound
+ refix rocket point blank
+ make fire animations non-syncronous
make MAX_PACKNAME define and error check against it
dissable qcache for release?
death/restart from loadgame
should mouse forward/backwards cause an auto-center?
unpackdir
load/save should not take full paths
ambient val on bmodels
dynamic lights on bmodels
finale
fix demo cycle problems
player corner clipping can take too long
option to report any unfilled pixels (jrmed2)
QuakeEd: goto point in command window
monsters shooting you underwater
-----------------------------------------
John Carmack's .plan for May 21, 1996
-----------------------------------------
* passage research
* fixed static sound start with -nosound
* optimized SV_CheckBottom for easy case
* sound optimization
* fixed kill coutners for world kills
* pop blood trails up higher
* key doors need to have touch removed
* added volume cvar to post scale all sound
* underscale sound
+ make hell knight only send one dynamic light out
+ tim6: wordtype typo
+ bmodel pvs issue in tim6
shotgun sound has dead time at end
damage amp in jrbase1 is pointless
easy to use "is drive readahead" test
listen server bugs
menu sounds
weapon offsets are wrong again
damage amplifier bug
check cd removal issues
tighten muzzle flash
phantom nail boxes in tim6
heads get stuck in ceilings sometimes
is lightning owner set properly? sandy got stuck using the lighting gun
-----------------------------------------
John Carmack's .plan for May 22, 1996
-----------------------------------------
* complete mouse configuration: m_pitch, m_yaw, m_forward, m_side
* m_filter sample smoothing option
* split non clearable parts of client_state_t into client_static_t
* clear items at level exit
* make hell knight only send one dynamic light out
* fix klook
* smooth centerview
* properly handle multiple inputs on a button, got rid of +forward2, +attack2
* exit game text screen
* gave name to unconnected clients
+ make torch animation client side!!!
is near Z clipping of alias models a bad idea?
allow multiple packfiles to override one another
better ideal pitch on slopes
mouse pitch/strafe combo
look at redbook audio
restart shouldn't drop console.
check for reloading the same level.
noambient flag for low memory systems
check zone memory allocation for bindings
differentiate reconnect
allow up/down in water
swimming frames
print bindings option
more edge friction?
reletive volumes for each ambient sound
explicit bindings for key up / key down?
teamplay totals
level name in solo scoreboard
-----------------------------------------
John Carmack's .plan for May 23, 1996
-----------------------------------------
* fixed body que
* Michael's client side alias animations
* Cash's fix for net connects
* fixed bobbing pvs problem
* fixed on-edge water grey problem
* fixed console keys generating double release events
* fix new pitching recenter
* changed powerup effect to dim light
partial screen updates
bloody face?
player induced meat chunks
seperate soudntrack files into seperate download?
check sighting into water
scroll water textures?
accumulate shotgun damage for powerup gibbing
override help/sell and finale in registered pak
victory music
append sample rate to music filenames to match best
rename jrstart to start
rename the last I_ functions to sys_
cache textures?
dim out screen when menus are up?
comment line on menu screens
del in save field
scan for all id1*.pak at startup
reclaim entity string space after spawning
change mins/maxs from floats to shorts in bspfile
cut vertex numbers to shorts?
marksurfaces to shorts?
remove view model in weird FOVs
ammo boxes not vanishing after reload
change dynamic light attenuations for tighter spheres
prog def for shareware only version?
-----------------------------------------
John Carmack's .plan for May 24, 1996
-----------------------------------------
* added lookspring back
* fixed client PVS for checkclient to be from viewofs
* fixed no sighting underwater
* no tarbaby damage to each other
* blob explosion core
* cleared prog stack on prog error, so recursion pverflow doesn't give
error
* fixed shalrath pain frame bug
* settriggermodel() now does angle, movetype, and solid init
* reduced client to server packet size
* trigger_changelevel fires targets
* end of level stuff
write a multi-platfirn C shell script for makefiles
360 jump trigger
shootable triggers?
take away all screen effects at completion
intermission code
more monster awareness to nearby carnage
reduce monster accuracy with invisibility
high res centerings
remove counters if cheating
rename cheat codes
generalize button0 button1 to button 0 - button 7
dynamically reconfigure listen clients
weapon movement with screen aspect is broken
option to not pause the game while console is down?
longer underwater time in easy skill?
set registered flag
menu sound (pause issue?)
remove shootable invisible triggers?
go over all the prog code
gamma in options menu
allow attack to respawn as well as jump?
put end screen in gfx
killed by box restart bug
video mode set bug
finish invisibility in deathmatch
intermission cameras
QuakeEd view pos button
save polygons by filling inside of sky volumes
load / save game
monster on plat issues
fish
shalrath
good lightning gun
escape twice to remove console
centerspeed not used?
optional center on mlook release
need bigger surface cache in higher res modes
run direct from cd working path option
cull distant sounds / partical effects on the server side?
change vector view_ofs to scalar viewheight
is the view model pitching backwards?
non-thrashing precache?
rename host_client to sv_client
check surface cache size performance figures
-----------------------------------------
John Carmack's .plan for May 25, 1996
-----------------------------------------
* fixed savegame / phantom box bug
* fixed underwater
* allow monsters to sight other just-damaged monsters as clients
* fixed lookspring
+ secrets and kills are backwards
tighten dlight radius/attenuation
copy center prints to console
the portal bug
cue music on completion
-----------------------------------------
John Carmack's .plan for May 26, 1996
-----------------------------------------
* flipped secret and kill plaques
* finish solo scoreboard
* finale
* optimized CL_CalcAmbientSounds
* optimized ResampleSfx
* reduced index sizes in .bsp files
* flood filled away outside sky fragments in bsp files
+ fix cache dir
demo loop issues
modify qbsp to allow entities only option to generate new triggers
randomize intermission spots
conditional bars in jrstart?
teleport sprite
fish
registration key
light bmodels
different quad damage effect
slipgate animations
lightning gun
general "no move" flag for player server C code
scaled texture lighting still has bugs
-----------------------------------------
John Carmack's .plan for May 27, 1996
-----------------------------------------
* moved ambient sound calculations to vis program
* moved file caching option to host_parms
* removed keybindings[1]
* added ambush flag to monsters
* made precache_* functions also return their parameter
* added precache_*2 functions for the registered version seperation
* modified qcc for -pak / -pak2 option to build shareware / registered
* random intermission_info selection
* automatic configuration saving
* angle 0 jump triggers
* fixed solo map name centering
artifact blinking before removal
deathmatch invisibility
blinking cursor for configuration screen
automatic config saving
sum quad damage for gibbing
shrink cl_entities to be cl_update_entities + cl_baselines
tim5 armor fall through bug
radius damage opening secret doors
only surpress autocenter when not on ground, not water check
clearing memory twice, allow baselevel to move if not connected
-----------------------------------------
John Carmack's .plan for May 28, 1996
-----------------------------------------
* QuakeEd: forbid dragging position / direction when textured
* constrained all file writes to the game directory to close security
holes
* allow multiple pak files intermixed with directory trees in the search
path
* full specification with: -basedir, -game, -cachepath, or -path
* moved demo list to cls, fixed efrag bug on second demo playback
+ bring up menu on any input during demos
+ do "loading" plaque from demo to demo
+ demo transition problem
+ automatic pak grabbing frame game dir
+ saving while in a trigger field kills it?
bump edicts again
acknowledge all - command line parms as they are accepted?
QuakeEd: don't explode if /qcache isn't present
qcc option to re-bsp models and maps
shambler lightning needs to be exactly positioned
joysticks
keybinding saving isn't right
claim all args from command line, or print warnings?
-----------------------------------------
John Carmack's .plan for May 29, 1996
-----------------------------------------
* any key in demos brings up menu
* demo transitions bring up loading plaque
* added timeout for loading plaque
* forced console away with loading plaque
* regular restart uses loading plaque
* disabled triggers now work properly after a savegame
* cleared edict before parsing to fix door beeping bug
* testing if(string) in qcc is not reliable, change to if (string != "")
* fixed loadgame cursor problem after saving a typed in name
* fixed dos savegame crash bug
* automatic pak0.pak pak1.pak etc loading from game directories
back to start after finishing each episode
episode finished items
ogre jumping frames
load bmodels by lumps for less memory damage?
does quake.qpr have to be in the root of the development directory?
crosshair option
wait on jump triggers?
drowning warning
cmd line load
reformat text on vid mode change
deathmatch level crossing
nightmare mode
death restart needs loading plaque
map name on savegame comment line
regular restart after a load game doesn't work
bound load pointer
swing up weapons
lightning gun
safe edict allocation failure
remove view model in wide FOV modes
give 9 faults
-----------------------------------------
John Carmack's .plan for May 30, 1996
-----------------------------------------
* automatic setting of the registered cvar
* demos play from pak file
* delay SCR_EndLoadingPlaque until after cache report
* added -minmemory parm for testing entry case
* stopped demo loop on demo load error
* stopped demo loop on connect and map
* zbuffer torch dropout bug fixed
* changed all remaining I_ functions to Sys_
* chase network issues
* colored light gel demo
remove bspleaktest from quake.qpr
can trigger_changelevel stuff two commands sometimes?
remove powerup shifts before intermission
gib code for the big monsters
give wizards (hknights?) need a muzzle flash
last_damage on zombie for cumulative gibs
new explosion
check min memory with large number of clients
better treading water
escape at a fulls creen console to restart demos?
allways run checkbox in options
instructions on configuration screen
fish attack
fish not leave water
finish invisibility
conditional registration message in jrstart
monsters die in slime
longer message triggers in jrstart
take dogs by ramp out in jrbase1
easier jump out of water
remove jsilly rbase1 bonus items
smotte light at start
loadgame from pulldown menu
amlev98 shambler needs red armor on easy level
for release: don't forget to remove "developer 1" cvar and bound cheats
-----------------------------------------
John Carmack's .plan for May 31, 1996
-----------------------------------------
* based load plaque removal on framecount instead of updates
* fixed crash bug with zerod notify lines
* load / save game menus converted to slots instead of file names
* end1 / end2 shareware / registered end screens
* triggered monsters no longer awake on invisible or notarget players
* broke out default.cfg from quake.rc
* fixed five way counter trigger message
* fixed push triggers to be frame rate independent
* translated spaces to _ in savegame comment to make stdio happy
* escape in keygrab now justs turns it off
* protected against recursive loop when Con_Printf inside
SCR_UpdateScreen
is model name caching being done correctly for the flushed world model
name?
centerprint message delay times
fade screen behind menus
gib big monsters
cl itemgettimes aren't saved. do they need to be?
weapon nudges on different view sizes
slime ambient sounds
center everything in higher res modes
got stuck by explosion near zombies
return to jrstart stuff
amtest98 needs new teleport textures
reset defaults option
reset to defaults option
track down load failed comments
why do menu sounds ometimes not work?
combo look/strafe
fix lightning
push trigger directionality wierdness
-----------------------------------------
John Carmack's .plan for Jun 01, 1996
-----------------------------------------
* tracked loadgame problem to stack overflow
* removed console cursor if key_dest != key_console
* ignore all autorepeat keys except backspace
* removed the extra unit of padding around bmodels that caused them to
clip
* build entity list for refresh instead of pushing efrags
* fixed cells not carrying from level to level
* nightmare skill item spawning is fixed
* old deathmatch rules by setting deathmatch 2
* options menu sliders
* moved gamma correction out of vid systems into view
+ shambler lightning needs pitch
- stop demo when menu comes up?
+ nightmare skill
+ integrate spritegen with
+ lookspring checkbox
+ make sure all menu options get saved
+ clear notify lines when in menu
increase surface cache size?
do knight swords do meat chunks?
centerspeed slider
-----------------------------------------
John Carmack's .plan for Jun 02, 1996
-----------------------------------------
* fish don't leave water
* merged new CD code
* shambler lightning pitch correct
* cut number of cells in half
* beam weapon entities only on client side
* fixed sliding death/no retart bug
* muzzle flash for wizards
+ regrab gold/silver keys on sbar
+ accumulate damage over a frame
+ dissalow more than two bindings to the same action
flymonster in wall test
jump cause center?
non-linear mouse movement reponse?
use 3 frames of gib3
start using pcx for all utilities
use slight for shambler setup
replace all item_weapon
lava damage on monsters
transition to intermission
warning sound on about to drown
don't end demo when player is in menu
-----------------------------------------
John Carmack's .plan for Jun 03, 1996
-----------------------------------------
* gate_boss, gate_episode
* item_sigil
* fixed demo transition forcing console and menu away
* setup for server transitions with connected clients
+ grab new explosions
+ console screen isn't coming down in deathmatch
+ non-pickup nails again
+ console cursor
+ check tim21
+ coop player starts
+ flip mouse y as a checkbox option
monsters aren't telefragged?
aim lighting gun up/down
connect while demo is running fucks up
sound for quad damage attack
accumulate damage
shoot flying gibs?
add new gibs
deathmatch intermission screen
talk message buffer doesn't erase with sized down screen
-----------------------------------------
John Carmack's .plan for Jun 04, 1996
-----------------------------------------
* allowed host_framerate to explicitly exceed the normal bounds
* reduced cls.state to ca_dedicated, ca_disconnected, ca_connected
* got consistant on when memory is dumped -- only at start, not end
* increaded MAX_VALUE to 1024 for longer text messages in entities
+ dlight bmodels
+ escape with console down brings up menu in addition to removing the
console
+ s_init should not be called on dedicated server
+ sound channel droppoff when lots going? combine torch sounds?
+ only allow two bindings to an action from menu
+ delete in options menu to unbind all
+ carry serverflags from level to level
+ make all items fire triggers
+ coop respawn
check dedicated, listen, loadgame, recorddemo, restarting, timedemo, and
demoloops with new server setup
make hit ground sound entity reletive
filling sky volumes causes rockets to explode on sky. fix somehow
make all teleporters have an "inactive" flag so an event makes them active
QuakeEd: top trace for shearing is at 2048, increase to 8192
remove relight for quad damage
remove leading K_ from keynames
player swimming frames
flash ammo counts after picking up bonus items?
shalrath doesn't attack zombies?
-----------------------------------------
John Carmack's .plan for Jun 05, 1996
-----------------------------------------
* coop is mostly working
* made snd_show give a total count
* combined static sounds of the same type
* demo record is now reord [demoname] [map]
* intermission takes all players in coop
* all players are invisible for coop intermission
* coop telefrag
* dead players are respawned on level change in coop
* don't bound clinet only systems to sys_ticrate
* fixed bug with monsters not sighting players after some level changes
* keys aren't picked up in coop
* moved colon on intermission
* cleared effects on player respawn
* fixed players 2+ not saving stuff on entering level
* coop colors follow from level to level
+ backspace / delete to remove bindings in options screen
+ coop quit doesn't send disconnect
+ super spike: s_spike
deathmatch weapon switching rules
blue sparks for lightning
check for splash before playing thump
center or scale menus in hi-res
change "gamma" to something else because of name clash with new djgpp
angle jump at demo start
coop should use solo scoreboard
alphabetize cvars / commands for tab completion
look for "backup past 0"
fix "gravity changed" message
powerup blinking
frozen demons?
timer on monsters mad at other players
-----------------------------------------
John Carmack's .plan for Jun 06, 1996
-----------------------------------------
* carry serverflags from level to level
* items fire targets when picked up
* different spikes for super, knight, and wiz
* removed requirement that triggers have a target
* removed all references to update_type
* lightning gun underwater explosion removes currentammo
* fixed bug in fat pvs that caused entire world to show on some frames
*.5 trigger only teleporters
* delete key bindings and prohibit tripple bindings
* blured savegame line
* don't allow cmd buffer sizebuf overflow
* behind menu fade
* seperate cursors for console and menu
* changed entity spawning to spawn-as-loaded
* move light entity hack to pogs
* draw sigils on status bar
+ remove self.running?
+ portal creation bug
+ make lightning gun more damaging again
+ sigil things on status bar
+ signon1 when at 1
+ weapon nudges
+ player faces
+ put talk.wav back in
+ wait before allowing intermission exit
+ teleport ambient sound?
quad damage liughtning explosion
menu4.wav for slider
make a generic signon stream and subdivide svs.signon more
velocity through levels?
all projectiles need a t ent
finish axe up
pickup weapons while firing bug
tick sound when pressing an unbound key?
escape back through menus
loadgame removes enemy state?
deathmatch intermission
texture animation speeds variable
use different exiting water sound
-----------------------------------------
John Carmack's .plan for Jun 07, 1996
-----------------------------------------
* three levels of gib velocity
* accumulate damage to allow gibbing with shotguns when powered up
* radius damage opens secrets
* get Michael's bmodel lighting code
* moved notarget check into progs code
* removed self.running from progs
* fixed weapon fov nudges, moved super shotgun
* status bar faces
* fixed single player loading plaques
* auto center if on bmodels
* sound and cd audio have big banners if not configured
* grunt muzzle flash sync with attack
* darkened blood
* dog head
* moved sound and cd init into non dedicated startup
* moved message printing to SUB_UseTargets (buttons get messages)
* moved delay to SUB_UseTargets so all activations can delay
* trigger_relay entity
* clear notify and center print when in menu
* added svs.changelevel_issued to prevent possible double changelevel
* changed NaN velocity / origin checks to warnings with clear to 0
+ take away stuff in jrstart
+ player head gib now has wrong view ofs
+ the tim5 bug
+ exactly position enforcer bolt launch
+ status bar faces
+ save game with powerups
+ monster target finding with prespawning is messed up
make activator a parm to use functions instead of a global
write a tool to search all maps for boolean queries on key/value pairs
reformat console with vid changes
start past skill select when returning to jrstart
connect to bad map, then to a good map gives no free clients
lava damage for monsters
shambler hand lightning
fade to intermission?
meat spray from axe attack
gamma prob at intermissions
make a model fullbright flag for torches
Add -safe for nosound, vidmode 0, exec default -nocdaudio, -nonet
kill zombies with axe
teamplay totals
doesn't auto center if head out of water
easier jump out of water
can't record net demos any more
solo status bar in dark font
key config names in dark font
print savegame comment line on save screen
are you sure? for quit
spawn explosion for lightning gun pop
invis face, invis + invuln face
monsters aren't allways facing target when firing
make the message prog functions take a destination: broadcast, static,
single
-----------------------------------------
John Carmack's .plan for Jun 08, 1996
-----------------------------------------
* teleport triggers UseTqargets
* fixed once triggers
* cleared static sounds between levels
* right justify options menu text
* cells became rockets on level cross
* changed teleport humm attenuation
* lookspring checkbox
* lookstrafe checkbox
* mouse y invert checkbox
* allways run checkbox
* made fov not a saved cvar
* wizspike and knightspike t ents
* take away stuff after completing an episode
* end of episode text
* head gib eye in floor bug
* blink cursor after messageboxes
+ everyone goes through the return teleporter in episode 4 jrstart
+ is bgm volume not working for cd?
+ make zoom a default configuration?
quad damage shotgun doesn't gib zombies
bonus item flashes
no menu sounds when disconnected
teleport ambient sound is no good
auto center when treading water
monster drowning
blaster mismatch warning is misformatted
axe kill zombies
new game loading plaque not showing
change zombie knock down code
reformat console on vid change
grenade launcher kicks too much
surface cache size in weird aspect ratio modes
prevent vid from allocing a big enough buffer to cause the next load to
fail
deathmatch invis eyes
monster reduced accuracy during invis
push for red super spikes
monster death bubbles underwater?
change name back to quad damage
disconnect from netgame isn't being sent
remove coop cheating
centerprint time reletive to length?
centerprint "entering ..." on level entry
-----------------------------------------
John Carmack's .plan for Jun 09, 1996
-----------------------------------------
* boss damage only by lightning
* enforcer bolt positioning
* all gibs centered
* lava splash
* boss stop throwing when player is dead
* fixed cl_backspeed for always run checkbox
* fixed double talk sound on triggers
* retain options menu cursor pos
* nightmare mode: no attack_finished limits, and five second pain interval
* triggered door message bug
* weeded out the unused entities in misc
* consolidated the four remaining sprite scripts into a single file
* modified sprgen to allow multiple grabs in a single file
* grabbed new explosions
+ level 1 gib needs work
+ is clipping detection for asm sound mixing working right?
+ items falling through other items bug
+ item on moving object respawn bug
+ models aren't checking dlight decay time
+ fix drop.map
+ status active/max is reversed
coop against boss
make qlumpy parse qc files as source?
combine particals and temp entities
fix gib rules
slime burn down
check edict return type
flash weapon icons
-----------------------------------------
John Carmack's .plan for Jun 10, 1996
-----------------------------------------
* more velocity on level 1 gib
* no pain sounds when invincible
* reduced lava coor shift
* deterministically spawn items on bmodels
* objects on blocked train no longer fall through
* hell knights need muzzle flash
* fixed flame offsets
* three levels of screen update
* erase exact lines of centerprint
* shareware non-finale transitions
* crash in _SV_TouchLinks
* escape puts away console
* laodgame from demo
* backpacks in coop
* floating gibs
* made beta3 CDs
"restart" in coop game drops to dos
word wrapping needs to count extra notify lines
telefrag isn't working sometimes
put a save plaque up when game is saving
demon died on top of player, left in air when moved
door sounds are broken
registered message shouldn't print at all
menu hotkeys should return to game on escape, not main menu
quickload shouldn't drop to console
quit/end game confirmation
split door is playing talk wav
sound looping pop
weapon frames should be decoupled from pain frames
door touch
take the k_ out of configs
velocity between levels
fish
shub
display key bindings command
deathmatch intermission
door messages
zombie statues
constant look down pitch bug is multi player related
permanent head tilt in coop mode
endscreen
connect while in demo is a bug
secret counts aren't broadcast
half dead player
frozen demon when dropped on player head
swimming frames
map e1m1 from a coop game gives loop qsocket used
all player model axe frames
sw version shouldn't allow path changes unless pop
-----------------------------------------
John Carmack's .plan for Jun 11, 1996
-----------------------------------------
* fixed the vis portalization order problem!
* allow attack to respawn as well as jump
* fixed faulting
* removed sprites directory and all references
* new qcc code for damage, should fix statue monsters
* new zombie pain code
hang on laptop sound init
boss needs to scan for coop players
centerprint powerup messages
change origin offset on fish for more fin
fish move fail on tcross surface should still allow xy move
fish should pull treading people underwater
make good spot for weapon change rules
8 bit sb output isn't using volume scale
slider sounds (esp for sound volume)
is e1m1 nailgun a trigger, or is it duplicated?
play sound or something after intermission will allow exit?
systems that translate blit need to update entire screen on palette
changes
-----------------------------------------
John Carmack's .plan for Jun 12, 1996
-----------------------------------------
* limit sound buffers to 4k, add extra mix points and mix full buffer
* fixed only registered trigger messages
* boss shouldn't spawn in deathmatch
* cleared move commands across level transitions
* 8 bit sound output volume scaling
* key configuration didn't work for changeweapon "impulse 10"
* options menu now resets string values of cvars
* talk drawing properly updates with sized down screen
* clear net connection ques on new connection
* f5 cheat or's in items
* removed error for unconnected buttons
* power up status bar faces
* armor number
* increased surface cache size
* end of game text
* menu up/down wrapping
* Sandy's menu suggestions
* jrstart alternate start point if returning
* play track 3 on intermission
* drop backpacks in coop
* got deathmatch invisibility working
clip into sky problem
demon smashed a player stuck into a sloping wall
no cheat in coop
automatic listen 2
talk message printing code
door sounds
check fly/float move code
check weapon changing while firing
check messagemode bound keys
"restart" on a client of a net game crashes
serverflag only trigger
gib head positioning
ability to specify cd track on demo recording
coop use solo scoreboard
two presses to exit intermission
deathmatch intermission screen
harder nightmare
name and color should be saved as cvars
undead players? fixed with new death code?
QuakeEd: trigger texture setting should be automatic
add colors to dm scoreboard
forward changelevel commands?
good POP
add "serial/modem configuration" menu
-----------------------------------------
John Carmack's .plan for Jun 13, 1996
-----------------------------------------
* slider sounds in menu
* trigger counter's don't require targets now
* fixed blaster mismatch warning message
* net start menu
* enter on full key config clears
* ogre gib
* quad damage face
* quad damage powerup cheat
* all gib heads
* removed K_ from key names
* bind with no parms reports the current value
can't give
try partical effect for teleporters
remove forward to server commands, use impulse for all?
old rule deathmatch option in menu
head in wall again
no cheat in coop
connect from demo
checksum pak directory for shareware version
shambler, wizard, ogre, voor heads
allow arrow key repeats?
item respawn on bmodel bug
pitch jerk at startup sometimes.
stuck heads bug
"no messages" option
axe weaponframe switch problem
make "dropped datagram" a developer only printf
player thump for ogre falling?
centerprint entering message
variable ceterprint wait time
double nightmare attacks
disallow new game during multiplayer?
no player crush to point
sinking into plats
coop through episode end
allow typing in savegame comment
fix talk message display code
keep commands in doubly linked MRU list to avoid linear scan?
multiplayer end of episodes hang?
centerview doesn't work in water
make sure all menu vars are in default.cfg
-----------------------------------------
John Carmack's .plan for Jun 14, 1996
-----------------------------------------
* removed all use of ai_dead in progs
* fixed nailgun fire order
* fixed double press to exit intermission
* fixed sliding dead player bug
* fix bonus item respawn on dropping plat
* zombies don't go down again if already on the way down
* fixed ambient sound looping problem
* show looping sounds lin soundlist
* combined rocket impact and explositon damage for reliable gibs
* fixed stuttering on dropped packets
* set teleport death size properly
easier gib from rockets for players
check telefrag for multi players stuck
is sound buffer not being properly cleared?
menu sounds
dissallow some server forward messages
internet reliable messages getting backed up?
check for hard packet overflows
flash items on status bar
stop using gfx.wad?
pickup rules
teleport into monsters bug
change away from axe bug
-----------------------------------------
John Carmack's .plan for Jun 15, 1996
-----------------------------------------
* handle escape special to prevent unbinding, and pass through to menus
* allow attack to restart as well as jump
* copied more cmd_argv through recursion
* boss lead player in hard mode
* removed blaster warnings on v4 cards
* blinking icons
* protect all cvars and commands that could be deathmatch cheats
* mvoed squish obit before fell to death
* fixed static sound false combining bug
* randomize axe swings
* never go into pain frames during weaponframes
* combined axe damage frames
* prevent bmodel teleport
* registered trigger only accepts players
* fixed activated door beeping
* fixed linked door messages
player crushed on e1m2 bug
make enter play slide noise on menu item adjustments
no head bobbing when dead
gib head look around
sink into rising plats
hack check for stuck
no such frame for gibbed dog head
bmodel dissapearing problem in e4m6
gib by damage isntead of negative health
option for listen servers to send out less packets?
crank menu3 sound up in volume
teleported monster flag
fill sky volumes differently?
shambler lightning hand frames
top line of console is not clipped correctly
door sounds / messages
weapon change rules
no gravity when in solid volume
double shots for monsters in nightmare mode
use multiple help screens
swim bubbles on forard move as well as jump
-----------------------------------------
John Carmack's .plan for Jun 16, 1996
-----------------------------------------
* removed hard coded number order for sbar
* moved palette and colormap to discrete lumps
* all help screens
* delay enter sound until after menu frame drawn
* delated recursive spawn death
* changed sound override rules to prevent monsters from eclipsing player
snds
* delay monster use reaction to guarantee sound after teleport
* added reliable_datagram
* fixed powerup respawning
* clear all of sv in spawn server
* heads bounce from telefrag
* fixed sizing bug with telefragging
* fixed force retouch bug
* bodyque objects get movetypes
player lightning source 16+ higher
check dead player coop transition
talk message needs to clear background
dissallow save from intermission
different default player colors?
kick clients off if unsent message gets too big
powerup flashes
demon and shambler rethink between claws
make boss face direction of throw
deathmatch weapon switching
double shots for monsters in nightmare mode
hot key menus should return to game with a single escape
item respawn with no pickup problem
floating heads
view center snapping
deathmatch intermission screen
directory checks
no frame bug with eyes
head floating in air
reliable broadcast
no exit option for persistant deathmatch servers
statue monsters
change pak files to pakS0 pakR0
triggered teleporters aren't working reliably
play sound when menu goes away
local sounds die sometimes
teleport effect
sounds on locked doors
pops at end of sound buffer: not mixing entire thing?
exp.tri spawn explosion frame
func_episodegate print messages
teleporter articles
fix "load failed" message
coop savegames
lightning gun on secret doors.
explain "kill" in manual
explain dedicated servers for large coop games in manual
-----------------------------------------
John Carmack's .plan for Jun 17, 1996
-----------------------------------------
* lighttning gun always activates secrets now
* fixed coop level to level bug when player is dead
* added final character animations
* only server can changelevel or restart
* fixed scoreboard color bars
* fixed spawn death
* changelevel from client faulted
* pop verification
* pak crc
* min shells on level transition
* voor spike
* coop use solo scoreboard
* teleport particals
* menu sounds
* frag.map
*? view center snapping
* scoreboard flags in higher res
* dm weapon pickup rules
* easier jump out of water
* cd track change at finale boss
* changed center print position for very long text messages
* force cd track on demos
+ -14 cells????
new sky filling
client enter time
end of e4 text crash
deathmatch game exits should stay at intermission
multiplayer boss ai
axe impact sound
rock ogre
stuck spawn
shambler recheck
coop savegames
talk message drawing
broke teleport sounds
stuck problem
water stuck
different pak naming structure
combine mo strip build
-----------------------------------------
John Carmack's .plan for Jun 18, 1996
-----------------------------------------
* fixed cells dropping below 0 with new lightning gun animation
* tighteneed teleport particals
* big hack to catch rare stuck occasions
* prevent save in intermission
* unbindall command
* fixed strip makefile
solo scoreboard not centered
jump out window while looking down looses auto-centering?
drop zombie, then crush it = unkillable zombie
changelevel with a dead server can take away shotgun/axe
knight first damage is too early
-----------------------------------------
John Carmack's .plan for Jun 19, 1996
-----------------------------------------
* fixed "no such frame" invis bug
* coop target tradeoff
* boss target tradeoff
* moved knight damage frames by one state each
* cl_signon messages are now developer only
* changed hell knight hit sound to sw data
* allow onolyregistered triggers to be used as killtriggers
* fencepost bug on name lengths
* added S_ExtraUpdate to Draw_FadeScreen
* ignore nointermission in deathmatch
* notifications for quit and new game
* created 0.9 SW RC distribution
tilde key doesn't type (unbindall)
monster counting bug
net can fuck up with a join
don't let demos change while in a menu
menu sounds still don't work in all cases
play menu sound on escape from menu
stuck spawn
disallow angle change during demo
nail messages say "fell to death"
rewrite buggy centering code
rewrite buggy point squashing code
color coded backpacks
no finale when deathmatching
fish
credits
-----------------------------------------
John Carmack's .plan for Jun 20, 1996
-----------------------------------------
(Shareware release, june 20th missing)
* fish do damage
* fixed lightning prog crash after killing boss
* added ctrl-alt-del check in key_event
* found problem with saving com port enabled in config files
* drop fish hp
* added samelevel flag for persistant servers
* auto advance to next level in ruled deathmatch
* deathmatch rankings show for 5 sec minimum
new game should bring up loading plaque
rewrite buggy screen management
big t dest
pain field on shub
stand in in center
turn off messages option
-----------------------------------------
John Carmack's .plan for Jul 01, 1996
-----------------------------------------
Quake is out. Finally.
I think the end product is damn good, and I am proud of my 18+ months of sweating over it. I hope lots of you enjoy the shareware levels enough to register :-)
While there will probably not be any about-faces in technical decisions, Quake will definately be growing and expanding over the next year or so as we gauge the user community reaction and release updates and the inevitable sequal.
Some work that will be going on in the near future:
Fixing any serious problems with Quake 0.91.
Release of tools and source code for user prog coding and level building.
Writing a new map editor for win/NT + open GL Porting Quake to native win32 + DirectX
Porting Quake on the metal to some 3D accelerator boards.
Writing a proper full radiosity replacement for the light utility.
Reasonable user suggested improvements (keep it in the discussion groups please, I get enough mail as it is)
Speed enhancements in the polygon model path.
Adding support for various gadgets -- HMDs, controllers, maybe other sound cards, etc.
And of course... The Next Generation Technology. (no, I won't give you a code name to talk about!)
Vacation? I don't need a vacation!
John Carmack Id Software
-----------------------------------------
John Carmack's .plan for Aug 02, 1996
-----------------------------------------
Here is The New Plan:
I copied off the quake codebase and set about doing some major improvements. The old v1.01 codebase will still be updated to fix bugs with the current version, but I didn't want to hold back from fixing things properly even if it involves some major changes.
I am focusing on the internet play aspect of the game. While I can lay out a grand clean-sheet-of-paper design, I have chosen to pursue something of a limited enough scope that I can expect to start testing it around the end of the month (august). I still have my grand plans for the future, but I want to get some stuff going NOW.
QuakeWorld.
The code I am developing right now is EXCLUSIVELY for internet play. It will be rolled back into the single player game sometime along the road to Quake 2 (or whatever it turns out to be called), but the experimental QuakeWorld release will consist of seperate programs for the client and the server. They will use the same data as the current registered quake, so the only thing that will be distributed is new executables (they will peacefully coexist with current quake).
There will be a single master server running here at id. Whenever anyone starts up a server, it will register itself with the master server, and whenever a client wants to start a game, it will inquire with the master to find out which servers are available.
Users will have a persistant account, and all frags on the entire internet will be logged. I want us to be able to give a global ranking order of everyone playing the game. You should be able to say, "I am one of the ten best QuakeWorld players in existance", and have the record to back it up. There are all sorts of other cool stats that we could mine out of the data: greatest frags/minute, longest uninterrupted quake game, cruelest to newbies, etc, etc.
For the time being, this is just my pet research project. The new exes will only work with registered Quake, so I can justify it as a registration incentive (don't pirate!).
If it looks feasable, I would like to see internet focused gaming become a justifiable biz direction for us. Its definately cool, but it is uncertain if people can actually make money at it. My halfway thought out proposal for a biz plan is that we let anyone play the game as an anonymous newbie to see if they like it, but to get their name registered and get on the ranking list, they need to pay $10 or so. Newbies would be automatically kicked from servers if a paying customer wants to get on. Sound reasonable?
Technical improvements.
The game physics is being reworked to make it faster and more uniform. Currently, a p90 dedicated server is about 50% loaded with eight players. The new network code causes a higher cpu load, so I am trying to at least overbalance that, and maybe make a little headway. A single p6-200 system should be able to run around ten simultanious eight player servers. Multiple servers running on a single machine will work a lot better with the master server automatically dealing with different port adresses behind the client's back.
A couple subtle features are actually going away. The automatic view tilting on slopes and stairs is buggy in v1.01, and over a couple hundred millisecond latancy connection, it doesn't usually start tilting until you are allready on a different surface, so I just ripped it out entirely. A few other non-crucial game behaviors are also being cut in the interest of making the physics easier to match on the client side.
I'm going to do a good chat mode.
Servers will have good access control lists. If somebody manages to piss off the entire community, we could even ban them at the master server.
The big difference is in the net code. While I can remember and justify all of my decisions about networking from DOOM through Quake, the bottom line is that I was working with the wrong basic assumptions for doing a good internet game. My original design was targeted at <200ms connection latencies. People that have a digital connection to the internet through a good provider get a pretty good game experience. Unfortunately, 99% of the world gets on with a slip or ppp connection over a modem, often through a crappy overcrowded ISP. This gives 300+ ms latencies, minimum. Client. User's modem. ISP's modem. Server. ISP's modem. User's modem. Client. God, that sucks.
Ok, I made a bad call. I have a T1 to my house, so I just wasn't familliar with PPP life. I'm adressing it now.
The first move was to scrap the current net code. It was based on a reliable stream as its original primitive (way back in qtest), then was retrofited to have an unreliable sideband to make internet play feasable. It was a big mess, so I took it out and shot it. The new code has the unreliable packet as its basic primitive, and all the complexities that entails is now visible to the main code instead of hidden under the net api. This is A Good Thing. Goodbye phantom unconnected players, messages not getting through, etc.
The next move was a straightforward attack on latency. The communications channel is not the only thing that contributes to a latent response, and there was some good ground to improve on.
In a perfect environment, the instant you provided any input (pressed a key, moved a mouse, etc) you would have feedback on the screen (or speaker) from the action.
In the real world, even single player games have latency.
A typical game loop goes something like: Read user input. Simulate the world. Render a new graphics scene. Repeat.
If the game is running 15 frames a second, that is 66 ms each frame. The user input will arive at a random point in the frame, so it will be an average of 33 ms before the input is even looked at. The input is then read, and 66 more ms pass before the result is actually displayed to the user, for a total of nearly 100 ms of latency, right on your desktop. (you can even count another 8 ms or so for raster refresh if you want to get picky).
The best way to adress that latency is to just make the game run faster if possible. If the screen was sized down so that the game ran 25 fps, the latency would be down to 60ms. There are a few other things that can be done to shave a bit more off, like short circuiting some late braeking information (like view angles) directly into the refresh stage, bypassing the simulation stage.
The bearing that this all has on net play, aside from setting an upper limit on performance, is that the current Quake servers have a similar frame cycle. They had to, to provide -listen server support. Even when you run a dedicated server, the model is still: fetch all input, process the world, send updates out to all clients. The default server framerate is 20 fps (50 ms). You can change this by adjusting the sys_ticrate cvar, but there are problems either way. If you ask for more fps from the server, you may get less latency, but you would almost certainly overcommit the bandwidth of a dialup link, resulting in all sorts of unwanted buffering in the routers and huge multi thousand ms latency times as things unclog (if they ever do).
The proper way to address this is by changing the server model from a game style loop to a fileserver/database style loop.
Instead of expecting everyone's messages to be dealt with at once, I now deal with each packet as it comes in. That player alone is moved forward in time, and a custom response is sent out in very short order. The rest of the objects in the world are spread out between the incoming packets. There are a lot of issues that that brings up. Time is no longer advancing uniformly for all objects in the world, which can cause a lot of problems.
It works, though! The average time from a packet ariving at the system to the time a response is sent back is down to under 4ms, as opposed to over 50 with the old dedicated servers.
Another side benefit is that the server never blindly sends packets out into the void, they must be specifically asked for (note that this is NOT a strict request/reply, because the client is streaming request without waiting for the replies).
I am going to be adding bandwidth estimation to help out modem links. If quake knows that a link is clogged up, it can choose not to send anything else, which is far, far better than letting the network buffer everything up or randomly drop packets. A dialup line can just say "never send more than 2000 bytes a second in datagrams", and while the update rate may drop in an overcommited situation, the latency will never pile up like it can with the current version of quake.
The biggest difference is the addition of client side movement simulation.
I am now allowing the client to guess at the results of the users movement until the authoritative response from the server comes through. This is a biiiig architectural change. The client now needs to know about solidity of objects, friction, gravity, etc. I am sad to see the elegent client-as-terminal setup go away, but I am practical above idealistic.
The server is still the final word, so the client is allways repredicting it's movement based off of the last known good message from the server.
There are still a lot of things I need to work out, but the basic results are as hoped for: even playing over a 200+ ms latency link, the player movement feels exactly like you are playing a single player game (under the right circumstances -- you can also get it to act rather weird at the moment).
The latency isn't gone, though. The client doesn't simulate other objects in the world, so you apear to run a lot closer to doors before they open, and most noticably, projectiles from your weapons seem to come out from where you were, instead of where you are, if you are strafing sideways while you shoot.
An interesting issue to watch when this gets out is that you won't be able to tell how long the latency to the server is based on your movement, but you will need to lead your opponents differently when shooting at them.
In a clean sheet of paper redesign, I would try to correct more of the discrepencies, but I think I am going to have a big enough improvement coming out of my current work to make a lot of people very happy.
-----------------------------------------
John Carmack's .plan for Aug 08, 1996
-----------------------------------------
Romero is now gone from id.
There will be no more grandiose statements about our future projects.
I can tell you what I am thinking, and what I am trying to acomplish, but all I promise is my best effort.
-----------------------------------------
John Carmack's .plan for Aug 10, 1996
-----------------------------------------
QuakeWorld structural addendum:
After hearing many arguments against the single master server, ranging from coherent and well reasoned to paranoid whining, I now agree that the single global master server isn't sufficient.
During the R&D phase, there will still be only the single server, but after all the kinks get worked out, I will allow a small number of sites to run private master servers. This will not be a general release, but only to properly licensed third parties. That will still allow me to collect my 100% coverage data, and it will prevent a single network/computer/software failure from preventing all QuakeWorld play.
QuakeWorld technical addendum:
I am reining in the client side prediction to a fairly minimal amount. It has too many negative effects in different circumstances. The problem of running away from or in front of your missiles was so bad that I considered simulating the missiles on the client side, but that is the second step on a slippery slope. If just the missiles were simulated, then a missile would fire through an enemy until the server informed you it exploded on them. Then you consider simulating interactions, but then you have to guess at other player inputs (which is hopeless)...
Lesson learned: Simulating 300 ms on the client side in a Quake style game is just out of the question. It probably works fine for flight sim or driving sims, but not in out twitch reaction games.
I am currently using client side simulation to smooth out the beat frequency interactions between server packet arrival and client frame times. In the shipping version of Quake, some latency was introduced on purpose to keep the displayed frame simulation time between the last two packets from the server so that the variability in arrival time could be smoothed out. In QuakeWorld, I am always starting with the most current packet, and using simulation to smooth out the variability. This <100ms of client side motion seems to be very manageable, and cuts out some real latency as well as doing the gueswork.
It looks like I am going to split the QuakeWorld client into multiple threads to reduce the avg 1/2 frame latency between input and packet sending. This is also a step towards building a multi-threaded Quake renderer, which will let multi-cpu NT machines render twice as fast. Lets hope the windows thread scheduler is decent...
I have been cutting down the message sizes a bit here and there as well. On a modem link, every couple bytes I save translates into a millisecond of latency saved. I cut an average of 17 bytes from the server to client and 8 from the client to server today.
-----------------------------------------
John Carmack's .plan for Aug 12, 1996
-----------------------------------------
Apogee's Prey team (and Duke's Levelord) leave 3drealms to work with Quake technology as Hipnotic Interactive.
:-)
-----------------------------------------
John Carmack's .plan for Aug 13, 1996
-----------------------------------------
I am considering increasing the default sv_friction value for QuakeWorld from 4 to 6 or 8.
It might take a little getting used to, but I think it gives more precise control for wide area network play.
If anyone wants to run some experiments with different friction levels on a current Quake server, I would be interested in hearing some feedback.
-----------------------------------------
John Carmack's .plan for Aug 17, 1996
-----------------------------------------
The remote server console commands are fully implemented for QuakeWorld.
To allow remote administration, the server must set the "password" cvar. By default, remote administration is turned off.
On a client, if you set the "password" cvar to the same value, you can issue "rcon" commands to the remote server :
rcon <server command> <arguments> ...
You can go to different levels, shut the server down, change cvars, ban people, etc. The output from the command is redirected over the net and will be echoed on the remote console.
You can also execute commands without even connecting to the server (if it was full) by setting the "rconadr" cvar to the full internet address (including port) of the system you want to administer.
2:00 in the morning:
My testarossa snapped another input shaft (the third time). damn dman damn. >1000 HP is bad for your drivetrain.
-----------------------------------------
John Carmack's .plan for Aug 18, 1996
-----------------------------------------
PACKET FILTERING
QuakeWorld supports two types of filtering: IP packet filtering and user id filtering. Userid filtering is the most convenient way to keep a specific person off of a server, but because anyone can create as many accounts as they want, a malicious user could just keep logging back in with a new account. If their ip address is banned, they won't be able to log on with any account from that computer. Unfortunately, most dialup accounts give a different ip address for each connection, so you may be forced to ban an entire subnet to keep a specific person off.
You can add or remove addresses from the filter list with:
addip <ip> removeip <ip>
The ip address is specified in dot format, and any unspecified digits will match any value, so you can specify an entire class C network with "addip 192.246.40".
Removeip will only remove an address specified exactly the same way. You cannot addip a subnet, then removeip a single host.
iplist Prints the current list of filters.
writeip Dumps "addip <ip>" commands to iplist.cfg so it can be execed at a later date. The filter lists are not saved and restored by default, because I beleive it would cause too much confusion.
filterban <0 or 1>
If 1 (the default), then ip addresses matching the current list will be prohibited from entering the game. This is the default setting.
If 0, then only addresses matching the list will be allowed. This lets you easily set up a private game, or a game that only allows players from your local network.
-----------------------------------------
John Carmack's .plan for Aug 22, 1996
-----------------------------------------
The rendition 3d accelerated version of Quake looks very good.
The image quality is significantly better than software - dithered, bilinear interpolated textures, and subpixel, subtexel polygon models.
It is faster than software even at 320*200, and at 512*384 it is almost twice as fast.
We do take a bit of a hit when we have to generate a lot of 16 bit surfaces, so occasionally the framerate gets a bit uneven, but overall it is a very solid improvement.
-----------------------------------------
John Carmack's .plan for Aug 27, 1996
-----------------------------------------
I haven't been working on QuakeWorld for the past week, because Michael has been working on the rendition port, which backburnered the win32 native port, which is a prerequisite for the QuakeWorld release.
Instead, I have been working on some better graphics code.
I now have a radiosity replacement for ligting quake maps. This will have a good effect on our Quake 2 maps, but probably won't be feasable for homebrew use.
I am finishing up a shadow volume pre-subdivision stage right now, and I need to do the resampling to quake light maps a bit better than I currently am, but I am very pleased with how it has turned out. A lot of the work will play into the next generation (NOT quake 2) game technology, for things like colored radiosity and dynamic shadows, but it is still cool just for calculating quake light maps.
Instead of placing lots of floating light entities, certain textures are tagged as being light emitters, so if you draw a flourecent light in the ceiling, it will automatically emit light from it's entire area. Lava emits light from the entire surface.
The simulation of the light is realistic now, instead of just hacked together. Light reflects off of surfaces, so ceilings will get lit even if there is no light pointing directly at them. The way light flows around corners looks eerily realistic.
I fixed the sky volumes so missiles dissapear inside them again, which was necessary to implement a new lighting feature: you can specify the position of the sun (or moon) in degrees, and the amount of light it will emit. The light then comes out completely parallel and at a constant intensity, passing through sky volumes to light up all outdoor areas in a consistant way. (this could be retrofited onto light.exe for people unable to run qrad)
Now for the bad news: a full size level takes several minutes to process on our quad alpha, and you MUST do a vis before running qrad (otherwise it would take about 1000 times longer). This wouldn't be out of reach even on mortal pentium systems (you can ask for a rough aproximation instead of the full job), except for the working set size. >100 megs. I'm NOT being wasteful here, either. I use a halved bit array for visibility interconnects, and sparse scaled shorts for energy transference.
I will release the source after we have been using it in production for a while, in case anyone wants to take a shot at reformulating it to be less memory intensive. It would be possible to get it down to 32 megs with some reduction in quality and a lot more wasted time, but then the run time would be obnoxious. I expect homebrew levels are going to have to stick with light.exe.
I am looking forward to doing a little more work on the other utilities as well. I want to break qbsp up into three seperate programs, so it will be able to run on machines with <32 megs of memory. Light can be sped up significantly if the PVS information is taken advantage of the way I did in qrad. I also have some theories on changing the algorithmic order of full vis processing time that I want to look into, especially now that the vis stage is put before production lighting.
-----------------------------------------
John Carmack's .plan for Sep 03, 1996
-----------------------------------------
I have been cleaning up the utilities for another release (soon) where everything is in Visual C++ project format. All of the common code is now included from the same place, so adding support for pcx or 3d studio files will only require changes in one file now for all utilities.
I made a few simple changes to qcc, and it now compiles FOUR TIMES FASTER! A recompile on my pp200 only takes 5 seconds now. I knew the linear search for defs was hurting, but if I had realized it was that dominant (and easy to fix) I would have fixed it a long time ago. All I had to do was move defs to the head of the search list each time they were looked for, so they would be found fast the next time. A big hash table might be somewhat faster still, but this was trivial to add.
Qbsp now uses floats instead of doubles by default, which makes it a good deal less of a memory pig, but it seems to give numerical accuracy problems in some cases.
I am splitting qbsp up into three seperate programs, which will make the memory requirements much smaller than they are now, and will help me be more numerically rigorous. This is going to take some time, so it won't be in the next code release.
-----------------------------------------
John Carmack's .plan for Sep 10, 1996
-----------------------------------------
Ok, I'm back on QuakeWorld.
While working on the new chat console, I fixed up a few little things with the regular console:
The pgup/pgdn keys now autorepeat.
The display does not pop down to the bottom when a new line is printed.
If you aren't at the bottom, a few ^ ^ ^ are printed on the last line.
There is now a message level you can set to cut down on the number of messages you get from the server (it saves a bit of bandwidth, too).
Level 0 gives everything.
Level 1 does not print item pickup messages.
Level 2 does not print player obituaries.
Level 3 does not print any messages from the server.
Level 4 does not print chat messages.
Player skin support is much nicer in QuakeWorld. There is a qw\skins directory, where you can just put a .pcx files for each skin. This will scale properly to hundreds of skins if you want to download them all.
I am still unsure of how much freedom I want to give with the skins.
One school of thought is that you just set the "skin" value to any name you want. If the other players don't have it, they fall back to the default.
The other school of thought is that skins are the restricted property of a specific clan, and only clan members get custom skins.
We have the ability to delegate specific information setting abilities to clan leaders. To join a clan, you have to get the leader to add the clan field to your user account. It is in the protected masterdata, so you can't edit it yourself.
-----------------------------------------
John Carmack's .plan for Sep 13, 1996
-----------------------------------------
There is a BETA distribution of 1.05 up on our ftp site.
ftp://ftp.idsoftware.com/idstuff/unsup/q105beta.exe
It includes quake.exe winded.exe progs.dat, and a zip of the latest .qc files. Back up your current version before messing with this, in case something is broken.
There is new source code available. ftp://ftp.idsoftware.com//idstuff/source/qutils.zip has win-32 compiled versions and VC++ projects for all of the quake command line utilities. The qbsp/light/vis programs have not been tested all that well here, because we still use a dec alpha running unix for most of our work, but it has been converted from doubles to floats, which should reduce the memory requirements quite a bit (and possibly cause more numeric instabilities...).
I also uploaded the DOOM ipxsetup / sersetup source code, which we haven't had on our ftp site before.
We are going to begin internal testing of a 1.05 Quake release tomorrow. When we upload for general release, we will also upload the current .qc source. The old qcc.tar.gz file is still present on the ftp site for the original .qc files for now.
-----------------------------------------
John Carmack's .plan for Sep 18, 1996
-----------------------------------------
We are now beta testing the first release of QuakeWorld.
It seems to be going well. The latency reductions are very real, and significantly improve control and responsiveness. There is some loss of smoothness in some cases, but you can tune the tradeoff to your liking.
I don't want everyone to think that it will instantly turn their 28.8 modem into a lan connection, but it should be solidly better than what we have in all circumstances.
On an internet connection that provides a stable flow of packets at a reasonably predictable rate, the gameplay is very good. Single packet drops are not even noticable, but dropping several packets at once or having a large variability in latency can still mess up a game.
The really cool ISPs will run QuakeWorld servers right at their pops, which should give 28.8 modem players 200 ms or less pings, providing a nearly flawless game.
-----------------------------------------
John Carmack's .plan for Sep 22, 1996
-----------------------------------------
I had a good time at the New York event -- it was fun to meet a lot of the web and clan guys, and there was some good fierce deathmatch action.
The QuakeWorld beta has revved a couple more times. When things work right, its great, but it seems like it only works right on about half the windows machines we try it on.
We have at least a week of beating on it before a public release.
-----------------------------------------
John Carmack's .plan for Oct 14, 1996
-----------------------------------------
I haven't been able to work on QuakeWorld for a week, and am unlikely to for a couple more days.
I've been working with the utilities and editor most of the time, but there has been a lot of other stuff going on lately that has kept me from being very productive.
-----------------------------------------
John Carmack's .plan for Oct 16, 1996
-----------------------------------------
The latest windows drivers from rendition fix the lousy GUI performance I had been complaining about to several people. While it may not be state-of-the-art 2d acceleration, it doesn't get in your way anymore. I do not consider windows performance to be a negative for rendition anymore.
-----------------------------------------
John Carmack's .plan for Nov 06, 1996
-----------------------------------------
Over the weekend, I ported Quake to open-gl / win32. It is a complete port with all features supported. Its not really practical right now on most consumer level equipment, but it runs ok on my intergraph glz-13t. It will work on the 3dlabs cards with their next driver release, but it really isn't very optimized yet, so don't expect anything really fast.
If you have high end hardware (intergraph realizm or dec powerstorm), it is pretty cool.
It is interesting to look at quake with hires 24 bit trilinear texture mapping. Sometimes you just don't even notice anything is different, but sometimes you can just stare at a scene for quite a while apreciating it.
Throwing bits and pixels at a static design certainly doesn't hurt, but it really doesn't bring it to a new level. I think I know where the next level is, but I'm just at the very beginning of the work on the next game architecture.
We have three mostly-done things to release: winquake, glquake, and quakeworld. No, I don't know when any of them are actually going to be released. Sorry.
QuakeWorld will actually download a new level for you now if you want to wait for it. Little models and sounds are a lot more practical to customize a server with, though.
Today I got cornered by the three level designers and an artist and they jumped all over me for working on new research instead of polishing the quake 2 tools. I guess the rest of my week is spoken for...
-----------------------------------------
John Carmack's .plan for Nov 23, 1996
-----------------------------------------
QuakeWorld: there is one physics problem that is preventing me from releasing: you get stuck sometimes in moving plats. I am having a hard time tracking it down, and I only get to work on it sporadically. I will spend a bit more time this weekend.
I have given the source for the master server to the QSpy guys and Disruptor to do with as they will (rankings, queries, etc). I know I won't have the time to support it like it deserves.
WinQuake: Michael is almost done.... We only have one reported serious problem, but there are still a list of little issues. We are finally feeling stable on windows. Not everyone may be able to get optimal settings, but SOMETHING should work almost everywhere.
I have an NT alpha machine on order, so we will be compiling for that as well. We aren't going to be able to spend time writing assembly language for it like the intel version has, but it should still run at a pretty decent clip. IBM was talking about getting us a power-PC machine, but I haven't heard back from them in a while. If one shows up, we will compile for that as well.
GLQuake: 3DFX is writing a mini-gl driver that will run glquake. We exepect very high performance. 3Dlabs is also working with me on improving some aspects of their open-GL performance. DynamicPictures sent me an Oxygen board, but glquake did a horrible crash and burn with their beta drivers. They are looking into it. DEC is sending me a powerstorm to work with, which should perform in the same ballpark as the intergraph realizm I did the development work on.
I do expect the 3dfx version to outperform the $5k+ professional gl cards, due to its higher fill rate and the fact that they can tune directly for quake. There are certainly valid reasons to buy $5k cards (24+ bit z buffers, 24 bit color, 1280 res, etc), but don't cry when a $300 card eats their lunch :-).
Because of the very fill-rate intensive nature of the way I implemented this version of glquake (using a blend pass to do the light maps), performance is not likely to be very good on any current generation consumer level board except 3dfx. The next generation of most vendors cards should have sufficient fill rate, if claims are to be believed. I may do a surface cached version just for the experience, in any case.
One little public rant: if you are a 3d chip designer, don't even THINK about not supporting true alpha blending. Screen door transparency is not a valid replacement. (this means YOU, matrox!)
IRIX-GLQuake: In two weeks, Ed Hutchins from SGI is coming down and we are going to port glQuake to irix. This will only run (reasonably) on systems with hardware texture mapping: O2, impact, RE2, and IR. No indys, extremes, etc. You could probably use them as dedicated servers, though.
D3DQuake: I started bringing up glquake on direct-3d a couple days ago. We are going to have a very nice, apples-to-apples api comparison here. It takes four times as much code to do anything in D3D as it does in OpenGL, but I will tune both to the best of my abilities and we shall see if there is any performance advantage to either API. We should be able to do comparisons on both 3DFX and 3DLabs hardware with both apis.
Quake utilities: I have split qbsp into two programs: qcsg and qbsp2. The processing is more robust, faster, and more memory frugal, but it currently generates rathar unoptimized maps. I will be writing a seperate map optimizer program soon to get the counts and size back to what they should be. I will probably do a new tool release after we get some more testing time on them. All of the utilities except qbsp now automatically use as many processors as you have if you are running on NT.
New stuff: The outline of the next generation game engine is beginning to take a bit of shape... I'm applying the lessons learned from quake and I have a long list of new things to try. I want to polish off all the outstanding quake stuff, then just go into hermit mode and work on the future while the other guys do Quake 2.
-----------------------------------------
John Carmack's .plan for Dec 13, 1996
-----------------------------------------
I need to clear up a little bit of really flagrant misinformation here:
Digital Sight and Sound, a dallas based system integrator, run a little blurb in their customer update newsletter that seriously misrepresents my experiences with intergraph and sgi hardware.
The title was "Z-13 versus sgi maximum impact, Z-13 wins" and it included statements like "After two weeks of grueling tests, the z-13 finished 2.5 times faster overall than the Maximum IMPCT... and at less than 1/2 the price!!!". This just isn't true, in saveral ways.
The last time I did side by side system testing involving SGI was pitting an SGI high impact against a intergraph GLZ1T. For heavy texture mapping applications like I was doing, the SGI high impact was three times faster than the intergraph. A realizm-13t is 2.5 times faster than the older GLZ1T, but a max impact is twice as fast as a high impact, so intergraph's best is still only half the speed of sgi's best workstation.
For the record, here is my impression of the 3D hardware I have worked with, unedited. Prices are for configurations as we would use them, not minimum.
Rendition Verite: ($150) It won't set any speed records, but it is an excellent value, the quake port is pretty good, and the risc architecture allows a lot of room for optimization.
3DFX: ($300) The highest textured fill rate of anything short of high end SGI hardware. 50 mpix, almost irrespective of options. You can't run a desktop or high res display on it, but at 640*480 it will stomp any other consumer level board, given equal optimization effort.
3dlabs permedia: ($300) Right now, this is the only well supported low end open GL card. and a good all around video card. The visual quality is hurt over most other 3d boards because the internal paths apear to be 15 bits, so you don't get dithered 24 bit color out of your blending operations. The fill rate is very sensitive to what options (z buffer, blending, color depth, etc) you have turned on. It peaks at over 40 mpix, but that is with no zbuffering and 4 bit textures. Realistic cases (16 bit textures, z buffering, perspective, bilinear) bring you down to around 10 mpix. There are some performance hangups in the drivers for heavy texturing applications, but 3dlabs is working on improving them.
The remaining boards are pretty much for professional use, not gaming.
Dynamic Pictures Oxygen: ($1500) I can't reccomend this to anyone doing texture mapping. It was slow and had rasterization errors.
3dlabs glint-TX: ($1000 - $3000) A reletively low fill rate (15 mpix peak, often a bit lower), but fairly stable in performance. Good GL drivers. Capable of supporting very high resolutions and amounts of texture memory. Available from several vendors at various (professional) price points.
Intergraph intense-3d: ($2000 - $5000). The fill rate still isn't very good (15 mpix), but it hardly cares when you turn on trilinear and blending. They rape you on the texture memory prices. I have gotten better performance (but not tons better) out of this than any glint card I have tested, but I have yet to use one of the dual TX boards. They are also incompatable with a lot of motherboards.
Intergraph realizm: ($7000 - $12000) The best graphics card you can buy for NT. As far as I know, you can't even get these seperately, you have to buy them as part of an intergraph system. The systems we use list around $30k each. The fill rate is 33 mpix, and very insensitive to options. You can add an optional geometry board to get very high transform/light throughput. DEC resells these as their Powerstorm T series cards for alpha systems.
SGI O2: ($10000, full system) The fill rate was better than intergraph's high end, but the triangle throughput was lower. I expect SGI will be able to optimize the porformance more in the coming months. It's not a knock-your-socks-off performer, but a real, fully configured system costs less than just the video option on a high end intergraph.
SGI IMPACT: ($25000 - $50000, full system) excellent fill rate and geometry rate. A very balanced, high performace system. The texture memories are very small (one or four megs), so it would probably be possible to contruct a pathalogical case that probably puts an intergraph ahead of it, but you would have to stretch. The prices are higher than intergraph, but only by about 30%.
SGI Infinite reality: ($100000+) Fill rate from hell. Polygons from hell. If you don't trip up on state changes, nothing will come within shouting distance of this system. You would expect that.
Our decision to go intergraph/NT over SGI/IRIX had a lot more to do with the NT vs IRIX part of it. I wish I could buy SGI hardware for NT. The last two SGI systems I did serious work on were messes of instability, but the O2 we just got last week does actually seem stable.
As far as the "which video card should I buy?" question from gamers, I'm afraid I can't give you a definitive answer. All of the first generation boards have some problem or another. The second generation, coming out within the next six months or so, will be more solidly positive.
-----------------------------------------
John Carmack's .plan for Dec 17, 1996
-----------------------------------------
QuakeWorld went up, got to around 4000 users, then the master server exploded.
Disrupter and cohorts are working on more robust code now.
If anyone did it on purpose, how about letting us know... (It wasn't all the people that tried %s as a name)
-----------------------------------------
John Carmack's .plan for Dec 21, 1996
-----------------------------------------
Wow. Apple bought NeXT. That really brightened my day.
I haven't generally been unhappy developing on NT, but I had been carrying a bit of sadness over several of the things we left behind when we moved from NEXTSTEP.
I wouldn't touch a mac for development right now, but if apple does The Right Thing with NeXT...
-----------------------------------------
John Carmack's .plan for Dec 23, 1996
-----------------------------------------
OpenGL vs Direct-3D
I am going to use this installment of my .plan file to get up on a soapbox about an important issue to me: 3D API. I get asked for my opinions about this often enough that it is time I just made a public statement. So here it is, my current position as of december '96...
While the rest of Id works on Quake 2, most of my effort is now focused on developing the next generation of game technology. This new generation of technology will be used by Id and other companies all the way through the year 2000, so there are some very important long term decisions to be made.
There are two viable contenders for low level 3D programming on win32: Direct-3D Immediate Mode, the new, designed for games API, and OpenGL, the workstation graphics API originally developed by SGI. They are both supported by microsoft, but D3D has been evangelized as the one true solution for games.
I have been using OpenGL for about six months now, and I have been very impressed by the design of the API, and especially it's ease of use. A month ago, I ported quake to OpenGL. It was an extremely pleasant experience. It didn't take long, the code was clean and simple, and it gave me a great testbed to rapidly try out new research ideas.
I started porting glquake to Direct-3D IM with the intent of learning the api and doing a fair comparison.
Well, I have learned enough about it. I'm not going to finish the port. I have better things to do with my time.
I am hoping that the vendors shipping second generation cards in the coming year can be convinced to support OpenGL. If this doesn't happen early on and there are capable cards that glquake does not run on, then I apologize, but I am taking a little stand in my little corner of the world with the hope of having some small influence on things that are going to effect us for many years to come.
Direct-3D IM is a horribly broken API. It inflicts great pain and suffering on the programmers using it, without returning any significant advantages. I don't think there is ANY market segment that D3D is apropriate for, OpenGL seems to work just fine for everything from quake to softimage. There is no good technical reason for the existance of D3D.
I'm sure D3D will suck less with each forthcoming version, but this is an oportunity to just bypass dragging the entire development community through the messy evolution of an ill-birthed API.
Best case: Microsoft integrates OpenGL with direct-x (probably calling it Direct-GL or something), ports D3D retained mode on top of GL, and tells everyone to forget they every heard of D3D immediate mode. Programmers have one good api, vendors have one driver to write, and the world is a better place.
To elaborate a bit:
"OpenGL" is either OpenGL 1.1 or OpenGL 1.0 with the common extensions. Raw OpenGL 1.0 has several holes in functionality.
"D3D" is Direct-3D Immediate Mode. D3D retained mode is a seperate issue. Retained mode has very valid reasons for existance. It is a good thing to have an api that lets you just load in model files and fly around without sweating the polygon details. Retained mode is going to be used by at least ten times as many programmers as immediate mode. On the other hand, the world class applications that really step to new levels are going to be done in an immediate mode graphics API. D3D-RM doesn't even really have to be tied to D3D-IM. It could be implemented to emit OpenGL code instead.
I don't particularly care about the software only implementations of either D3D or OpenGL. I haven't done serious research here, but I think D3D has a real edge, because it was originally designed for software rendering and much optimization effort has been focused there. COSMO GL is attempting to compete there, but I feel the effort is misguided. Software rasterizers will still exist to support the lowest common denominator, but soon all game development will be targeted at hardware rasterization, so that's where effort should be focused.
The primary importance of a 3D API to game developers is as an interface to the wide variety of 3D hardware that is emerging. If there was one compatable line of hardware that did what we wanted and covered 90+ percent of the target market, I wouldn't even want a 3D API for production use, I would be writing straight to the metal, just like I allways have with pure software schemes. I would still want a 3D API for research and tool development, but it wouldn't matter if it wasn't a mainstream solution.
Because I am expecting the 3D accelerator market to be fairly fragmented for the forseeable future, I need an API to write to, with individual drivers for each brand of hardware. OpenGL has been maturing in the workstation market for many years now, allways with a hardware focus. We have exisiting proof that it scales just great from a $300 permedia card all the way to a $250,000 loaded infinite reality system.
All of the game oriented PC 3D hardware basically came into existance in the last year. Because of the frantic nature of the PC world, we may be getting stuck with a first guess API and driver model which isn't all that good.
The things that matter with an API are: functionality, performance, driver coverage, and ease of use.
Both APIs cover the important functionality. There shouldn't be any real argument about that. GL supports some additional esoteric features that I am unlikely to use (or are unlikely to be supported by hardware -- same effect). D3D actually has a couple nice features that I would like to see moved to GL (specular blend at each vertex, color key transparancy, and no clipping hints), which brings up the extensions issue. GL can be extended by the driver, but because D3D imposes a layer between the driver and the API, microsoft is the only one that can extend D3D.
My conclusion about performance is that there is not going to be any significant performance difference (< 10%) between properly written OpenGL and D3D drivers for several years at least. There are some arguments that gl will scale better to very high end hardware because it doesn't need to build any intermediate structures, but you could use tiny sub cache sized execute buffers in d3d and acheive reasonably similar results (or build complex hardware just to suit D3D -- ack!). There are also arguments from the other side that the vertex pools in d3d will save work on geometry bound applications, but you can do the same thing with vertex arrays in GL.
Currently, there are more drivers avaialble for D3D than OpenGL on the consumer level boards. I hope we can change this. A serious problem is that there are no D3D conformance tests, and the documentation is very poor, so the existing drivers aren't exactly uniform in their functionality. OpenGL has an established set of conformance tests, so there is no argument about exactly how things are supposed to work. OpenGL offers two levels of drivers that can be written: mini client drivers and installable client drivers. A MCD is a simple, robust exporting of hardware rasterization capabilities. An ICD is basically a full replacement for the API that lets hardware accelerate or extend any piece of GL without any overhead.
The overriding reason why GL is so much better than D3D has to do with ease of use. GL is easy to use and fun to experiment with. D3D is not (ahem). You can make sample GL programs with a single page of code. I think D3D has managed to make the worst possible interface choice at every oportunity. COM. Expandable structs passed to functions. Execute buffers. Some of these choices were made so that the API would be able to gracefully expand in the future, but who cares about having an API that can grow if you have forced it to be painful to use now and forever after? Many things that are a single line of GL code require half a page of D3D code to allocate a structure, set a size, fill something in, call a COM routine, then extract the result.
Ease of use is damn important. If you can program something in half the time, you can ship earlier or explore more aproaches. A clean, readable coding interface also makes it easier to find / prevent bugs.
GL's interface is procedural: You perform operations by calling gl functions to pass vertex data and specify primitives.
[code]
glBegin (GL_TRIANGLES);
glVertex (0,0,0);
glVertex (1,1,0);
glVertex (2,0,0);
glEnd ();
[/code]
D3D's interface is by execute buffers: You build a structure containing vertex data and commands, and pass the entire thing with a single call. On the surface, this apears to be an efficiency improvement for D3D, because it gets rid of a lot of procedure call overhead. In reality, it is a gigantic pain-in-the-ass.
[code]
(psuedo code, and incomplete)
v = &buffer.vertexes[0];
v->x = 0; v->y = 0; v->z = 0;
v++;
v->x = 1; v->y = 1; v->z = 0;
v++;
v->x = 2; v->y = 0; v->z = 0;
c = &buffer.commands;
c->operation = DRAW_TRIANGLE;
c->vertexes[0] = 0;
c->vertexes[1] = 1;
c->vertexes[2] = 2;
IssueExecuteBuffer (buffer);
[/code]
If I included the complete code to actually lock, build, and issue an execute buffer here, you would think I was choosing some pathologically slanted case to make D3D look bad.
You wouldn't actually make an execute buffer with a single triangle in it, or your performance would be dreadfull. The idea is to build up a large batch of commands so that you pass lots of work to D3D with a single procedure call.
A problem with that is that the optimal definition of "large" and "lots" varies depending on what hardware you are using, but instead of leaving that up to the driver, the application programmer has to know what is best for every hardware situation.
You can cover some of the messy work with macros, but that brings its own set of problems. The only way I can see to make D3D generally usable is to create your own procedural interface that buffers commands up into one or more execute buffers and flushes when needed. But why bother, when there is this other nifty procedural API allready there...
With OpenGL, you can get something working with simple, straightforward code, then if it is warranted, you can convert to display lists or vertex arrays for max performance (although the difference usually isn't that large). This is the right way of doing things -- like converting your crucial functions to assembly language after doing all your development in C.
With D3D, you have to do everything the painful way from the beginning. Like writing a complete program in assembly language, taking many times longer, missing chances for algorithmic improvements, etc. And then finding out it doesn't even go faster.
I am going to be programming with a 3D API every day for many years to come. I want something that helps me, rather than gets in my way.
John Carmack
Id Software
-----------------------------------------
John Carmack's .plan for Jan 12, 1997
-----------------------------------------
We now have someone officially in charge of quake/quakeworld unix ports:
Dave 'Zoid' Kirsch. (zoid@threewave.com)
Direct any correspondance about the linux ports of quake to him. If any other unix vendors would like to have quake ported to their environments, set up an equipment loan with him. This is a volenteer position, so don't give him a hard time.
-----------------------------------------
John Carmack's .plan for Jan 22, 1997
-----------------------------------------
A preliminary release of glquake and the 3dfx driver has been put on our ftp site in the idstuff/unsup directory.
one hour later...
3dfx gave me a new vxd for glquake that fixes crashing problems on some pentium pro systems. glquake1.zip now contains the current files.
-----------------------------------------
John Carmack's .plan for Jan 25, 1997
-----------------------------------------
I fixed a couple problems in glquake with some addon stuff -- the goats demo actually showed up a crashing problem with 256*256 textures, and the rangers demos showed that players with non-player models and colormaps were all messed up.
I'll make another release when 3dfx has their next driver drop finished.
-----------------------------------------
John Carmack's .plan for Jan 31, 1997
-----------------------------------------
I went down to the ferrari dealership today with Ann and American with the intention of buying a new f355 as a "sensible" car (in contrast to my other twin turbo monstrosities).
There was an F40 sitting out front.
Umm. Sensible car. F40. Sensible car. F40. Hmmmm.
American played the evil conscience and Ann played the good conscience. For a while. Then Ann got infected by the Dark Side. :-) I bought the F40.
Ok, I now have too many ferraris. I have a plan to fix that. This hasn't gone through all the legal crap necessary yet, but it is my intention to give away my first ferrari as the grand prize of a Quake tournement.
Yes, I am serious.
DO NOT SEND ME ANY MAIL ABOUT THIS! When we have more to say about it, we will make a formal announcement. This would be a good time to start a heated debate in the newsgroups about what the fairest tourney rules would be, though.
The specs:
1987 328 GTS with turbocharged engine.
Feature in the january, 1994 issue of Turbo magazine.
Made 360 HP at the rear wheels on a chassis dyno.
New engine (I melted the first one...), new paint.
The interior could use a little work.
I plan to also include enough cash to cover tax and insurance.
-----------------------------------------
John Carmack's .plan for Feb 05, 1997
-----------------------------------------
I've been following the ferrari discussion on r.g.c.q, and here are a couple clarifications:
The final rounds will definately be one on one deathmatch over a LAN, probably double elimination.
The big question is how do we get the number of people who would like to take a shot at it down to a reasonable (64?) number of people for a proper tournement.
We don't know yet where the final rounds are going to be held, either. E3? #Quakecon? A microsoft event? A totally specific event?
I know that whatever we come up with won't be 100% fair to our entire customer base, but it would be really unfortunate if some people got bitter over it.
-----------------------------------------
John Carmack's .plan for Feb 13, 1997
-----------------------------------------
There will be a new QuakeWorld release soon with a very different interface and several improvements.
On the topic of network gaming, I have a couple questions that I would like answered by anyone with the required technical knowledge:
If I understand correctly, UDP headers are currently not compressed over PPP links like TCP headers commonly are. During a network game (or internet phone or videoconference), all the packets are just between two addresses, but tons of bandwidth (and corresponding latency) is eaten by sending full headers. Has anyone developed UDP header compression extensions, and if so, how common are they?
On a lower hardware level, have any modem manufacturers considered optimizing interfaces or protocols for lower packet latency? Once again, if I understand correctly, modem transmission involves a level of packetizing for modulation/protocol/compression work that is underneath the serial interface seen by a computer. Do partially filled buffers sit idle for a time if there isn't a continuous stream of data, or do they immediately flush when a byte failes to clock in? It would be nice to be able to explicitly syncronize with UDP packets, because a realtime game running within bandwidth limits is going to send and receive discrete packets interspersed with idle time, which is not really what modems are optimized for now. I know that modem data compression is generally a bad thing for low latency connections right now, but if the modem could be signaled to flush at UDP boundaries, it would probably become a positive thing. I'm sure there are also things that could be done at the modulation/protocol layer to improve latency at some cost in bandwidth or error rate, but I have no idea if it could be done compatably.
It would be an interesting point of differentiation for ISP's or modem manufacturers if an "optimize for games" checkbox was available somewhere. Even at the os level, a "max packets buffered" option would be valuable to keep packets from piling up when bandwidth is overcommited.
Comments?
-----------------------------------------
John Carmack's .plan for Feb 18, 1997
-----------------------------------------
Ferrari update.
Unless someone can come up with a compelling objection, I think we have an end plan now:
Intergraph is currently holding a very cool clan tournement that will have the finals at E3. The ClanRing guys are managing the tournement, and intergraph is picking up the bill. It's going to be great, with clan banners and projection screens. They allready have a pretty good prize lineup of cash and systems.
The current proposal is to expand the event to include one on one deathmatches in addition to the clan fights. The grand prize will be my 328. I don't think it would go over too well to have a ferrari as clan property :-)
Intergraph is willing to pay for travel and accomadations for twelve to sixteen contestants.
We still need to figure out how to select the finalists. The idea is that we can tell all the press types about the event now, and have a last-minute runnoff of some sort the month before E3. That way even people who aren't usually active on the internet can find out about it through a magazine and take a shot at it.
-----------------------------------------
John Carmack's .plan for Feb 19, 1997
-----------------------------------------
I made significant improvements to the scalability of QuakeWorld.
You have probably noticed how a 16 player game is a lot worse over a modem than a 4 player game, even when you aren't around the action. All of those factors are now gone, so big games play a lot better now, and I have bumped the maximum players to 32. There aren't really any levels that will be reasonable with that many players, but if someone wants to create one, it should be possible now. (obviously you are still going to go slow if all 32 people decide to congregate in the same room)
The next release of QuakeWorld is going to be completely incompatable with the current release. Both the client and server have changed protocols, and the fate of the master server is still undecided.
I have one more really significant thing to try with the network protocols that I should probably hold up the release for, because it would be yet another incompatable change.
-----------------------------------------
John Carmack's .plan for Feb 23, 1997
-----------------------------------------
I took an entire day and a half off from work to spend some quality time with my F40 at Texas World Speedway. I had a lovely moment pitching my quarter million dollar car off the track backwards (no harm done), but otherwise I had a great time. Back to work now.
I should have updates of both QuakeWorld and GlQuake within a week. QW is crapping out at 27 players for some reason (rather difficult to debug...), and glquake needs a fix for the hipnotic level pack to work.
My next project is to define a new rendering architecture that will clean a bunch of stuff up and allow me to combine regular quake, glquake, and a windows version of vquake into a single executable. I plan on doing the development work with QW, so I won't be stepping on Michael or Cash's toes as I go hacking and slashing through the codebase. If everything goes well, that will become the new guts of Quake 2, and I will probably also release a unified Quake 1 executable.
-----------------------------------------
John Carmack's .plan for Mar 05, 1997
-----------------------------------------
I have been soooo busy lately. (yes, thats my excuse for not having new glquake and quakeworld release yet)
Glquake works on the hipnotic pak now, but I still need to make a couple more changes. 3dfx has a new opengl.dll nearing completion.
QuakeWorld should be released soon, synced with a new version of qspy and CTF progs. I thought I was in feature freeze a week ago, but day before yesterday I couldn't help myself and I started implementing the last big thing I know how to do to improve the net play. I might as well get all the incompatible stuff done in one release, rather than breaking things again later. This release should make a lot of people happy.
We got our new SGI Origin-2000 server, and I have been tuning all the tools to work better on eight processors. I really wanted to buy an intel system running NT, but all of the big (8-16 processor) systems from Sequent, Data General, and NCR have incredibly brain damaged price tags assigned to them (in the neighborhood of $40,000 PER PROCESSOR).
I finally broke down and wrote a 3D model painting program for our artists. We have tried four seperate commercial programs, and none of them really do what we want. If you want something done right... The models in Quake2 are looking better in a lot of ways -- more polygons, more texels, more exacting texture work, smoother movement, etc.
I have been doing some cleanup work on our map editor, because we may wind up bundling it with an OpenGL accelerator card in the reletively near future. The terms aren't worked out yet, so I can't say much more.
Aaron Seeler from midway is coming down in a few days and we are going to start thrashing out the architecture for Quake on the N64. I'm really looking forward to that -- I have some clever things in mind that should really leverage the N64 hardware and deliver an awesome game. All of our previous cart ports have been just trying (and failing sometimes...) to equal a mid range PC game experience, but Quake on the N64 should be a lot closer to the vquake / glquake versions than the vanilla software version that most people are fammiliar with.
-----------------------------------------
John Carmack's .plan for Mar 11, 1997
-----------------------------------------
The new glquake is on ftp.idsoftware.com. It should work for both the hipnotic pack and the upcoming rogue pack.
/idstuff/unsup/glq3_11.zip
-----------------------------------------
John Carmack's .plan for Mar 12, 1997
-----------------------------------------
Ok, this really pisses me off:
[code]Date: Wed, 12 Mar 1997 13:32:43 +0100
From: "%Peter Lawrenz" <%Lawrenz@bln.de>
Reply-To: %Lawrenz@bln.de
X-Mailer: Mozilla 3.01 (Win95; I)
To: johnc@idsoftware.com
Subject: Re: E3/Ferrari Event PRESS RELEASE
Will Bryant wrote:
> John Carmack, the co-founder of id Software and creator of
> the game QUAKE, will give away one of his four Ferraris as
> the grand prize: a red 1987 Ferrari 328 GTS with a Norwood
> Autocraft turbo conversion. According to Carmack, donating
> the Ferrari is his way of giving something back to the game
> enthusiasts that have brought him success. "I bought my first
> Ferrari after the success of Wolfenstein-3D," said Carmack.
> "DOOM and QUAKE have bought three more. Four Ferraris is too
> many for me. Rather than sell off one of them or stick it in a
> warehouse, I'm going to give it back to the gamers that brought
> it to me in the first place. The king of this QUAKE deathmatch
> is going to get a really cool crown." "This is the biggest,
> hottest tournament ever for PC gamers, and we encourage all QUAKE
> players to give it their best shot," said Wade.
>
> Tournament Registration and Contact Information QUAKE players who
> want to participate in the tournament can register online at
> <mpog.com/e3signup.html>, anytime between March 24 and
> May 2, 1997. Participation is limited to US residents 18 years of
> age and older, other rules are posted on the Website.
Hi,
the two above statements contradict each other. How many ferraries
would you have been able to buy if the sales of your games were limited
to U.S. only ? It seems to me that you don't value your oversea
customers at all and invite them to pirate future ID software products.
I paid for Doom I, Doom II and Quake and since this press release I am
not really sure if I should spend any more money on ID software.
There are other companies that might value their customers for real.
-----
_|_ Sometimes you lose, sometimes the others win.
---*--O--*---
O O X Peter Lawrenz (DFV3772)
X X Quake + Spaceorb360 = FUN
X lawrenz@remove.the.%.from.the.reply-to
FreeFall-> FreeBag-> FreeBeer[/code]
AAAAARRRRGGGHHHH!!! I have gotten a few of these "you suck cuz I can't win your car" type of messages.
I'm giving the car away over the objections of my lawyer and my girlfriend just because I think it will be a cool and memorable thing to do. I want it to be as fair as possible, but I just don't have the time to spare to try and manage it myself. In any case, it wouldn't be fair to everyone no matter what we did. Sure, it would be great if we could fly the 10,000 interested people to a football stadium and have thousands of simultanious double ellimination bouts, but it just isn't going to happen.
I pushed against the "us citizans" clause, asking if it would be reasonable to bring a couple people from europe or austrailia, or even just allow some people that covered their own expenses, but aparently we would have to deal with the local laws in each country that an entrant comes from, which just wouldn't be reasonable for us. Even dealing with canada (as we have found out recently with Christian) can be a big pain in the ass.
Its just a fact of life that locality is an issue. We can't treat the entire world the same. Go convince another company that "might value their customers for real" to give you a ferrari.
-----------------------------------------
John Carmack's .plan for Mar 13, 1997
-----------------------------------------
Here is a technical issue to be discussed:
I am strongly considering dropping qc in Quake 2 in favor of exporting most of the game logic to a seperate .dll file. This wasn't an option when we had to support dos, but I think it is the correct choice now.
There are a lot of issues involved with this.
As everyone who has tried to do anything serious with qc knows, it has its limitations (ahem). I could improve the language, or just adopt a real language like java, but the simplest thing to do would be just use native code.
It would definately be more efficient as a dll. As we do more sophisticated game logic, efficiency becomes more and more important. For simple deathmatch modifications this wouldn't be a big deal, but for full size game levels it will likely be at least a 5% to 10% overall speed improvement.
It would be non-portable. I am dreading the reaction to this from the linux community. Game modifications would have to be compiled seperately for each target architecture (windows, linux, irix, etc). I do still intend to have the client be generic to all mods (but more flexible than Q1), so it is really only an issue for servers.
There are security concerns. I suppose to a world that embraces Active-X, this isn't really an issue, but binary code patches still spook me.
You would actually need a compiler to hack quake. For the serious people, this isn't an issue, but it would cut out a number of people that currently enjoy hacking quake. I have a strange mixture of pride and shame when I think about the people that have actually started learning programming in my crappy little qc language.
You could debug your patch in a real debugger! Yipee!
-----------------------------------------
John Carmack's .plan for Mar 18, 1997
-----------------------------------------
I have gotten a significant amount of response on the Quake 2 extension mechanism. I do read everything that comes my way (I can't respond to all of it, though), and I have learned a few things from the mail.
Nothing is set in stone yet, but it is still looking like a dll is going to be the primary interface. I have been seriously considering a java interface, but the tradeoffs (time spent implementing takes away from something else..) just don't quite add up. Other options, like enhancing qc or using other languages like perl have very remote chances.
One of the primary reasons is that you can allways build UP - put more functionality on top of a dll, but you can't allways build DOWN - accessing the registry from java for instance.
For Id Software to develop a game, a dll will be most efficient. We have more cpu power, and we can debug it more easily. We are directing significant effort towards making Quake 2 a better GAME, as well as just a better mutliplayer virtual world. Quake 1 was pretty messed up from a game standpoint, and we don't plan on doing that again.
What I can offer the qc hacking crowd is a public release of the qc interface and interpreter code from Quake 1 when Quake 2 is released. The user community can then bolt things together so that there can be one publicly trusted DLL that executes an updated and modified qc language for portable, secure add ons.
I really do care about portability, but it is just one factor that needs to be balanced against all the others. Things just aren't clear cut.
Speaking of portability, to remove the guesswork that goes on, here are my current opinions on the various platforms:
Win32
Win32 rules the world. You are sticking your head in the sand if you think otherwise. The upside is that windows really doesn't suck nowdays. Win 95 / NT 4.0 are pretty decent systems for what they are targeted at. I currently develop mostly on NT, and Quake 2 will almost certainly be delivered on win32 first. Our games should run as well as possible in NT, we won't require any '95 only features.
Dos
We are not going to do another dos game. No amount of flaming hate mail is going to change my mind on this (PLEASE don't!). The advantages of good TCP/IP support, dynamic linking, powerfull virtual memory, device drivers, etc, are just too much to overcome. Yes, all of those can be provided under dos in various ways, but it just isn't worth it.
Linux
I consider linux the second most important platform after win32 for id. From a biz standpoint it would be ludicrous to place it even on par with mac or os/2, but for our types of games that are designed to be hacked, linux has a big plus: the highest hacker to user ratio of any os. I don't personally develop on linux, because I do my unixy things with NEXTSTEP, but I have a lot of technical respect for it.
MacOS
From a money making standpoint, the only OS other than win32 that matters, and it doesn't matter all that much. We have professional ports done to MacOS instead of unsupported hack ports, which is a mixed blessing. They come out a lot later (still waiting for quake..), but are more full featured. I have zero respect for the MacOS on a technical basis. They just stood still and let microsoft run right over them from waaay behind. I wouldn't develop on it.
OS/2
A native OS/2 port of any of our products is unlikely. We just don't care enough, and we are unwilling to take time away from anything else.
SGI
I don't particularly care for IRIX as a development environment (compared to NT or NEXTSTEP), but SGI has the coolest hardware to run GL apps on. Safe to assume future IRIX ports, but its not exactly a top priority.
AIX / OSF / HPUX / SOLARIS
I wouldn't start a port to any of these, but if a trusted party (Zoid) wanted to do them, I probably wouldn't object.
BeOS
I bought a BeBox because I am a solid believer in SMP, and I like clean, from-scratch systems. I was left fairly non plussed by it. Yes, it is lean and mean and does a couple things better than any other OS I have seen, but I just don't see any dramatic advantages to it over, say, NEXTSTEP. Lion (the company doing the mac quake port) has a BeOS port of quake sort of working, and have my full support in releasing it, but it will be strictly an act of charity on their part, so don't expect too much.
Plan9
I spent a few months running Plan9. It has an achingly elegent internal structure, but a user interface that has been asleep for the past decade. I had an older version of quake dedicated server running on it (don't ask me for it - I lost it somewhere) and I was writing a civilized window manager for it in my spare time, but my spare time turned out to be only a couple hours a month, and it just got prioritized out of existance.
NEXTSTEP
My faviorite environment. NT and linux both have advantages in some areas, but if they were on equal footing I would choose NEXTSTEP hands down. It has all the power of unix (there are lots of things I miss in NT), the best UI (IMHO, of cource), and it just makes sense on so many more levels than windows. Yes, you can make windows do anything you want to if you have enough time to beat on it, but you can come out of it feeling like you just walked through a sewer.
In the real world, things aren't on equal footing, and I do most of my work on NT now. I hold out hope that it may not stay that way. If apple Does The Right Thing with rhapsody, I will be behind them as much as I can. NEXTSTEP needs a couple things to support games properly (video mode changing and low level sound access). If apple/next will provide them, I will personally port our current win32 products over.
If I can convince apple to do a good hardware accelerated OpenGL in rhapsody, I would be very likely to give my win NT machine the cold shoulder and do future development on rhapsody. (I really don't need Quickdraw3D evangelists preaching to me right now, thank you)
-----------------------------------------
John Carmack's .plan for Mar 23, 1997
-----------------------------------------
Someone ran into my F40 in the parking lot, then took off. Words cannot do justice to how I feel right now.
If anyone knows a tall white male in the dallas area that now has red paint and carbon fibre on their tan pickup truck, turn the bastard in!
-----------------------------------------
John Carmack's .plan for Apr 02, 1997
-----------------------------------------
The second generation QuakeWorld is out and live now.
We will probably release a couple bug fix versions over the next week or so as problems are reported.
Overall, I'm pleased with the results - I think I have delivered very solid improvements in game play. I certainly learned a lot along the way. If you have anything resembling a decent connection, you should be able to play a good game. A server with a 400+ ms ping and 10% packet loss is still not going to play all that great, but you should just avoid those.
The packet size is about as small as it is going to get for the general cases. Any more shrinkage will have to be game-specific compression, like the specialized nail update.
I can make doors and plats move smoothly, but it will take a good chunk more development. This will probably be done for quake 2.
I have it all set up to clip movement against other players during prediction, but I probably need a day or two to finish it. I'm not confidant that I'll get to that anytime soon, though.
I really want to get client side demo recording and more spectator mode options (see through player's eyes, chase cam, etc), but I just don't have the time right now.
The next major upgrade will be a quakeworld that can run in software and OpenGL modes. A verite module will come later.
This combination (QW networking and switchable rendering) will be the base that we move all of our Quake 2 work over to.
-----------------------------------------
John Carmack's .plan for Apr 04, 1997
-----------------------------------------
Ok, the current OpenGL code no longer scales the status bar and console. You can stop complaining now. The next release will be the consolidated rendering code for quakeworld. I'm not sure when I will be able to make a standalone version.
The consolidated quake will also be available on NT-alpha as well as x86. If you have a powerstorm-T card, glquake works pretty good. Glint and oxygen cards don't work well enough, but the normal quake software version should work fine. We may get a little bit of asm code written for the software version.
-----------------------------------------
John Carmack's .plan for Apr 08, 1997
-----------------------------------------
Technical note:
Instanced brush models are not going to be well supported in the next release of the quake engine. Instanced bmodels are models that are created as an entirely seperate map, like the health boxes and ammo boxes. We are not going to use any instanced bmodels in q2, but I will probably leave the drawing code intact for backwards compatability. You will NOT be able to movement clip against them as a bsp hull, however. You could still use a solid bounding box if you really have to have one, though.
Brush models built as part of the world, like doors and plats, will remain with full functionality.
Instanced bmodels never were lighted properly, and a lot of code gets simpler with this decision.
-----------------------------------------
John Carmack's .plan for Apr 09, 1997
-----------------------------------------
Would anyone complain if I took out lookstrafe and keyboard look modifier? I'm cleaning up code, and I don't know of anyone that ever uses those features.
update: ok, +klook has it's supporters. Anyone for lookstrafe?
-----------------------------------------
John Carmack's .plan for Apr 24, 1997
-----------------------------------------
The consolidated QuakeWorld client has been working pretty well. I've been playing deathmatch with it in GL mode for the past week. There are still a number of things to do on it, and I haven't been working on it for a while due to higher priority tasks. A lot of other non-graphics things have changed in the new architecture as well.
It is really cool to be able to switch between software and gl without even restarting the game. We will be testing Quake 2 extensively in GL and even doing some specific development for it. My current wild guess is that about 15% of quake 2 customers will run the OpenGL version (there will be several cards coming out this year that are fast enough, besides just 3dfx), so it is definately a factor in our decisions.
The verite renderer will still be supported in quake 2, but it won't have the special features of glquake. (it will still have it's own custom features like anti-aliasing, though)
There is a very cool new surprise feature for you in the next gl release :)
----
For the past several days I have been working on a new version of qbsp that should be dramatically more robust for "crazy" maps. Raven is aproaching completion on Hexen 2, and they have a couple problems in their maps, so this is my top priority.
I figured out something about CSG / BSP operations that had been kicking around in the back of my head for almost two years now. The seperate (and epsilon issue prone) CSG phase is not needed at all if you directly contruct a BSP tree from volumes instead of from polygons. I have that part working, but there is just so much work in the tools that getting the rest of the stuff working again is taking quite a lot of effort.
I will make another tool release when things calm down, but understandably that is about at the bottom of my priority list.
-----------------------------------------
John Carmack's .plan for Apr 28, 1997
-----------------------------------------
I'm sure you have all heard about the 3drealms / quake deal by now. It took a long time to get everything nailed down, but it should be worth it.
The "quake 2" terminology is a little confusing. They have all the quake / glquake / quakeworld stuff right now, but they will be picking up the quake 2 codebase after we finish it.
I'm quite excited about this - it should be a very complimentary arrangement. We would never have done a game like Duke at id, but there are many valid styles of design that are mutually exclusive. Todd and the rest of the Duke team are hard working developers with a pretty clear vision of what they want. It happens to be different than our vision, but the market is plenty big enough for both of them.
-----------------------------------------
John Carmack's .plan for May 06, 1997
-----------------------------------------
Brian Hook has been hired as our new programmer.
Brian wrote the glide API for 3dfx, worked on CosmoGL, and wrote a book on 3d programming that he is now horribly embarrased about.
-----------------------------------------
John Carmack's .plan for May 12, 1997
-----------------------------------------
I have gotten several emails speculating that there will now be a native glide port of quake. Here is the straight answer:
I have considered a glide port many times (especially now that the rendering code is in a dll), but I allways reach the conclusion that it wouldn't be justified.
On the plus side, it could get a 10%-15% speedup over the OpenGL driver without going through too many contortions. Primarily by saving transforms for the lightmap pass and doing tightly packed vertex arrays for the enemy models.
The big drawback is that every codepath that gets added holds back future innovation. Just having software and gl is a lot of work, and I have allready commited to verite support. This is a difficult point for some people to understand, but it is crucially important. The more places I need to rewrite a feature, the less likely I am to put it in. If I only had the opengl version to worry about, Quake 2 would be so much cooler..
-----------------------------------------
John Carmack's .plan for May 14, 1997
-----------------------------------------
As some of you may know, a port of Quake was demod at apple's WWDC. Here is the full info:
A couple weeks ago, I got an email saying: "Hey! We heard you are porting quake for WWDC!". I replied: "Uh, first I've heard of it.. I was planning on supporting Quake 2 on it late this year.."
Well, I stole some time and went ahead and did it (mostly last weekend - running tight!). I'm quite happy with how it turned out, and I'm glad it made it for the demos.
It is actually a port of the current research QuakeWorld-merging-into-Quake2 codebase, so it only plays network games at the moment.
It is running through 24 bit display postscript, and doesn't have the assembly language compiled in, so don't believe anyone that says it was running faster than under windows. It was a fast demo system. There is a good chance that it will be a bit faster then win32 when I am done with it, because the direct-to-screen API doesn't require all the locks and unlocks of Direct Draw, and the sound access will avoid the DirectSound penalties, but basically they should be the same.
98% of the support I need for games is present in rhapsody, and now that there is an existing game for it, the remaining decisions can be rationally guided.
I am still going to press the OpenGL issue, which is going to be crucial for future generations of games.
I am definately going to support Quake 2 on rhapsody. I may make a public release of the QuakeWorld demo, but I will probably wait until we get the full screen api working. Omnigroup has a little qspy-like openstep program that we can use with it.
-----------------------------------------
John Carmack's .plan for May 22, 1997
-----------------------------------------
Bad news.
It looks like this is when "unsupported" really becomes unsupported
Glquake and QuakeWorld were fun to do, but keeping the datasets compatable with quake 1 really has held me back a lot. I badly wanted to get one more release out, but circumstances have forced me to finally ireversibly break with compatability, and I just don't have the time to devote any effort to a stagnant codebase. You probably wont see any more releases from Id until hexen 2 ships. Sorry.
I have given Zoid and Jack Mathews free license to extend and upgrade the QuakeWorld codebase from the last released revision, so this may actually mean that QW receives more attention than I was able to give it.
On the bright side, the new bsp format will offer some great new capabilities that will be apreciated by all:
Greater robustness. Only one bsp tree is built, and no surfaces are generated that weren't part of the map brushes.
No fixed clipping hull restrictions. You can now set any mins/maxs you like.
You can tell the texture that a trace clips on in the game, so different surface attributes are now possible.
Textures are no longer stored in the bsp file.
Full color lightmaps for glquake. The "surprise" that I mentioned before was colored lighting hacked into glquake in a way that didn't require a change in the format, but this is better.
If any hard-core add on hackers can present a serious case for additional modifications to the bsp file, now is the time to let me know.
-----------------------------------------
John Carmack's .plan for Jun 19, 1997
-----------------------------------------
I'm pretty damn pleased with things right now.
We are just buttoning up the E3 demo stuff, and it looks really good. It is clearly way alpha meterial, but people should be able to project where it is going.
The timing is a bit inconvenient for us, because we still aren't quite through with converting all the .qc work that Cash did over to straight C code in the new engine. The monsters are just barely functional enough to show, with none of the new behavior in. If E3 was a week or two later, the demos would almost be real playtesting.
Q2 is going to be far and away the highest quality product id has ever done. There are new engine features, but the strength of the product is going to be how everything is fitted together with great care. (don't worry, next year will be radical new technology all over again)
--
Sound is being improved in a number of ways.
All source samples are 22 khz / 16 bit, and you can restart the sound system for different quality levels without exiting the game. high quality sound will require more memory than the base 16 meg system. The system can automatically convert to 11 khz / 8 bit sounds, but we are probably going to include a seperate directory with offline converted versions, which should be slightly higher quality. Homebrew paatches don't need to bother.
Sounds can now travel with a moving object. No dopler effects, but it positions properly. (well, spatialization is a bit fucked this very instant, but not for long)
I finally got around to tracking down the little bug with looping sounds causing pops.
I have intentions to do three more things with the sound engine, but the realistic odds are that they won't all make it in:
Voice over network. I definately don't have time to do a super-compressed version, but I can probably hack something in that the T1 players would have fun with.
Radiosity sound solution. Its obvious in retrospect, but it was a "eureka!" thought for me when I realized that the same functions that govern the transport of light for radiosity also apply to sound. I have research plans for next-generation technology that include surface reflection spectrums and modeling the speed of sound waves, but I think I can get a simplified solution into Q2 to provide an ambient soundscape with the same level of detail as the lightmaps. I'm a little concerned about the memory footprint of it, but I'm going to give it a shot.
Syncronized, streaming sound from disk. Special events and movie demos won't need to precache gigantic sounds, and they can rely on the timing.
--
Q2 has a generalized inventory structure and status display that should be adaptable to just about anything anyone wants to do in a TC.
--
On saturday, I give my 328 away at E3. I know that there were lots of issues with the contest, and to be honest, I probably wouldn't have done the nationwide contest if I could have forseen all the hassle (I could have just given it away at #quakecon..), but the finals should still be really cool. It just wasn't possible to make the contest "completely fair". Not possible at all. In any case, I don't think anyone will deny that the finalists are some of the best quake players around.
--
I doubt I can convey just how well things are going here. Things probably look a little odd from the outside, but our work should speak for itself. I have been breaking into spontanious smiles lately just thinking about how cool things are (of course, that could just be a sleep deprivation effect..).
We have a totally kick-ass team here.
We are on schedule. (no shit!)
We are doing a great product.
Everyone watch out!
-----------------------------------------
John Carmack's .plan for Jun 22, 1997
-----------------------------------------
Ok, I'm finally updating my .plans at the top like everyone else..
E3 was interesting, and the tournement went extremely well.
You would think that the top deathmatchers would be an evenly matched group, seperated by mere milliseconds of response time, and the matches would be close.
Its not like that at all. There are master players. And there is Thresh.
We were watching him play with our jaws hanging open. I don't think he was killed a single time in the finals. He did things we had never seen before. It was amazing to watch.
I feel a lot better about the contest now, because even if the sixteen finalists weren't necessarily the sixteen best players due to internet issues, I do think that the grand prize winner IS the best single player.
The level of sportsmanship was gratifying, especially given the stakes. No sore losers, no tantrums. Everyone was cool.
After the finals, a japanese champion (highroller) asked for a match with Thresh. I expected him to pass, considering the pressure of the tournement, but he happily accepted, and delivered an eighty-something to negative-three beating (which was accepted with good grace).
I don't see much point to any more big tournements until a few more of these mutant superpowered deathmatchers show up..
As far as everything else at E3 goes, I saw a bunch of good looking games, but I am fairly confident of two things:
Nobody is going to eclipse Quake 2 this christmas. Different tradeoffs are being made that will appeal to different people, and there are going to be other products that are at least playing in the same league, but Q2 should be at the top of the pile, at least by the standards we judge games. Several licensees will be picking up all the Q2 features for their early '98 products, so games should get even better then. (ok, I guess that is just my cautious, long-winded way of saying Q2 will rule..)
Some notable companies are going to ship longer after us than they are expecting to, or make severe compromises. I wouldn't advise holding your breath waiting for the quoted release dates. Relax, and let the developers get things out in due time.
Ugh. I haven't coded in three days. Withdrawal.
-----------------------------------------
John Carmack's .plan for Jun 25, 1997
-----------------------------------------
We got the new processors running in our big compute server today.
We are now running 16 180mhz r10000 processors in an origin2000. Six months ago, that would have been on the list of the top 500 supercomputing systems in the world. I bet they weren't expecting many game companies. :)
Some comparative timings (in seconds):
[code]
mips = 180 mhz R10000, 1meg secondary cache
intel = 200 mhz ppro, 512k secondary cache
alpha = 433 mhz 21164a, 2meg secondary cache
qvis3 on cashspace:
cpus mips intel alpha
---- ---- ---- ----
1 608 905 470
2 309 459
3 208 308
4 158 233
8 81
12 57
16 43
[/code]
(14 to 1 scalability on 16 cpus, and that's including the IO!)
The timings vary somewhat on other tools - qrad3 stresses the main memory a lot harder, and the intel system doesn't scale as well, but I have found these times to be fairly representative. Alpha is almost twice as fast as intel, and mips is in between.
None of these processors are absolutely top of the line - you can get 195 mhz r10k with 4meg L2, 300 mhz PII, and 600 mhz 21164a. Because my codes are highly scalable, we were better off buing more processors at a lower price, rather than the absolute fastest available.
Some comments on the cost of speed:
A 4 cpu pentium pro with plenty of memory can be had for around $20k from bargain integrators. Most of our Quake licensees have one of these.
For about $60k you can get a 4 cpu, 466 mhz alphaserver 4100. Ion Storm has one of these, and it is twice as fast as a quad intel, and a bit faster than six of our mips processors.
That level of performance is where you run into a wall in terms of cost.
To go beyond that with intel processors, you need to go to one of the "enterprise" systems from sequent, data general, ncr, tandem, etc. There are several 8 and 16 processor systems available, and the NUMA systems from sequent and DG theoretically scale to very large numbers of CPUS (32+). The prices are totally fucked. Up to $40k PER CPU! Absolutely stupid.
The only larger alpha systems are the 8200/8400 series from dec, which go up to 12 processors at around $30k per cpu. We almost bought an 8400 over a year ago when there was talk of being able to run NT on it.
Other options are the high end sun servers (but sparc's aren't much faster than intel) and the convex/hp systems (which wasn't shipping when we purchased).
We settled on the SGI origin systems because it ran my codes well, is scalable to very large numbers of processors (128), and the cost was only about $20k per cpu. We can also add Infinite Reality graphics systems if we want to.
Within a couple years, I'm sure that someone will make a plug-in SCI board for intel systems, and you will be able to cobble together NUMA systems for under $10k a cpu, but right now the SGI is the most suitable thing for us.
I have been asked a few times if Quake will ever use multiple processors. You can allways run a dedicated server on one cpu and connect to it to gain some benefit, but that's not very convenient, doesn't help much, and is useless for internet play.
It's waaaay down on the priority list, but I have a cool scheme that would let me make multiple copies of the software rendering dll and frame pipeline the renderers. Response is cut by half and the frame rate would double for two cpus, but pipelining more than a frame would be a bad idea (you would get lag on your own system).
I wouldn't count on it, but some day I might take a break from serious work and hack it in.
There is no convenient way to use multiple processors with the hardware accelerated versions, except to run the server on a seperate cpu.
That will probably be an issue that needs to be addressed in the lifespan of the next generation technology. Eventually people are going to start sticking multiple cpus (or multiple thread issue systems sharing resources) on a single chip, and it will become a consumer level item. I'm looking forward to it.
-----------------------------------------
John Carmack's .plan for Jul 03, 1997
-----------------------------------------
This little note was issued to a lot of magazines by microsoft recently. Just for the record, they have NOT contacted us about any meetings.
All the various dramas in this skit haven't quite settled down, but it looks like microsoft is going to consciously do The Wrong Thing, because of politcal issues. Sigh.
Our goal was to get the NT OpenGL MCD driver model released for win-95, so IHVs could easily make robust, high performance, fully compliant OpenGL implementations. Microsoft has squashed that. Flushed their own (good) work down the toilet.
The two remaining options are to have vendors create full ICD opengl implementations, or game specific mini-drivers.
Full ICD drivers are provided by intergraph, 3dlabs, real3d, and others, and can run on both NT and 95 (with code changes). Microsoft still supports this, and any vendor can create one, but it is a lot of work to get the provided ICD code up to par, and bug prone. On the plus side, non-game tools like level editors can take full advantage of them.
Minidrivers certainly work fine - we have functional ones for 3dfx and powerVR, and they have the possibility of providing slightly better performance than fully compliant drivers, but partial implementations are going to cause problems in the future.
We will see some of both types of drivers over the next year, and Quake 2 should work fine with either. We also intend to have Quake 2 show up on several unix systems that supports OpenGL, and I still hope that rhapsody will include OpenGL support (we'll probably port a mini-drivers if we can't get real support).
Once again, we won't be idiotic and crusade off a cliff, but we don't have to blindly follow microsoft every time they make a bad call.
Subject: Microsoft D3D vs. OpenGL
Author: Julie Whitehead at Internet
Date: 6/23/97 10:01 AM
Dear Editor,
You may be aware of a press release that was issued On June 12, by Chris Hecker, former MS employee and developer of D3D [sic]. The statement asks Microsoft to develop a stonger link between D3D and OGL.The press release, was signed by several game developers representing the top tier 3-D game developers. Microsoft is dedicated to maintaining an active relationship with its DirectX developers. In response to this request Microsoft will host the developers included in the statement at a developers roundtable in July. The purpose of the roundtable is to openly consolidate input and feedback from developers. Tentative date for the roundtable is immediately following Meltdown, July 18.
Direct3D is Microsoft's recommended API for game developers with more than 100 developers using Direct3D as the defacto consumer API. OGL is widely regarded as a professional API designed for high precision applications such as CAD, CAM, etc. Our hope is that this round table will provide Microsoft with the feedback required to evolve our 3D APIs in a way that delivers the best platform for our developers.
If you have any questions or wish to speak with a Microsoft spokesperson, please let me know.
Julie Whitehead
-----------------------------------------
John Carmack's .plan for Jul 07, 1997
-----------------------------------------
The quality of Quake's software has been a topic of some discussion lately. I avoid IRC like the plague, but I usually hear about the big issues.
Quake has bugs. I freely acknowledge it, and I regret them. However, Quake 1 is no longer being actively developed, and any remaining bugs are unlikely to be fixed. We would still like to be aware of all the problems, so we can try to avoid them in Quake 2.
At last year's #quakecon, there was talk about setting up a bug list maintained by a member of the user community. That would have been great. Maybe it will happen for Quake 2.
The idea of some cover up or active deception regarding software quality is insulting.
To state my life .plan in a single sentance: "I want to write the best software I can". There isn't even a close second place. My judgement and my work are up for open criticism (I welcome insightfull commentary), but I do get offended when ulterior motives are implied.
Some cynical people think that every activity must revolve around the mighty dollar, and anyone saying otherwise is just attempting to delude the public. I will probably never be able to convince them that isn't allways the case, but I do have the satisfaction of knowing that I live in a less dingy world than they do.
I want bug free software. I also want software that runs at infinite speed, takes no bandwidth, is flexible enough to do anything, and was finished yesterday.
Every day I make decisions to let something stand and move on, rather than continuing until it is "perfect". Often, I really WANT to keep working on it, but other things have risen to the top of the priority que, and demand my attention.
"Good software" is a complex metric of many, many dimensions. There are sweet spots of functionality, quality, efficiancy and timeliness that I aim for, but fundamentally YOU CAN'T HAVE EVERYTHING.
A common thought is that if we just hired more programmers, we could make the product "better".
It's possible we aren't at our exactly optimal team size, but I'm pretty confidant we are close.
For any given project, there is some team size beyond which adding more people will actually cause things to take LONGER. This is due to loss of efficiency from chopping up problems, communication overhead, and just plain entropy. It's even easier to reduce quality by adding people.
I contend that the max programming team size for Id is very small.
For instance, sometimes I need to make a change in the editor, the utilities, and the game all at once to get a new feature in. If we had the task split up among three seperate programmers, it would take FAR longer to go through a few new revs to debug a feature. As it is, I just go do it all myself. I originated all the code in every aspect of the project, so I have a global scope of knowledge that just wouldn't be possible with an army of programmers dicing up the problems. One global insight is worth a half dozen local ones.
Cash and Brian assist me quite a lot, but there is a definite, very small, limit to how many assistants are worthwhile. I think we are pretty close to optimal with the current team.
In the end, things will be done when the are done, and they should be pretty good. :)
A related topic from recent experience:
Anatomy of a mis-feature:
----
As anyone who has ever disected it knows, Quake's triangle model format is a mess. Any time during Quake's development that I had to go back and work with it, I allways walked over to Michael and said "Ohmygod I hate our model format!'. I didn't have time to change it, though. After quake's release, I WANTED to change it, especially when I was doing glquake, but we were then the proud owners of a legacy data situation.
The principle reason for the mess is a feature.
Automatic animation is a feature that I trace all the way back to our side-scroller days, when we wanted simple ways to get tile graphics to automatically cycle through animations without having to programatically each object through its frames.
I thought, "Hmm. That should be a great feature for Quake, because it will allow more motion without any network bandwidth."
So, we added groups of frames and groups of skins, and a couple ways to control the timing and syncronization. It all works as designed, but parsing the file format and determining the current frames was gross.
In the end, we only used auto-frame-animation for torches, and we didn't use auto-skin-animation at all (Rogue did in mission pak 2, though).
Ah well, someone might use the feature for something, and its allready finished, so no harm done, right?
Wrong. There are a half dozen or so good features that are apropriate to add to the triangle models in a quake technology framework, but the couple times that I started doing the research for some of them, I allways balked at having to work with the existing model format.
The addition of a feature early on caused other (more important) features to not be developed.
Well, we have a new model format for Quake 2 now. Its a ton simpler, manages more bits of precision, includes the gl data, and is easy to extend for a couple new features I am considering. It doesn't have auto-animation.
This seems like an easy case - almost anyone would ditch auto-animation for, say, mesh level of detail, or multi-part models. The important point is that the cost of adding a feature isn't just the time it takes to code it. The cost also includes the addition of an obsticle to future expansion.
Sure, any given feature list can be implemented, given enough coding time. But in addition to coming out late, you will usually wind up with a codebase that is so fragile that new ideas that should be dead-simple wind up taking longer and longer to work into the tangled existing web.
The trick is to pick the features that don't fight each other. The problem is that the feature that you pass on will allways be SOMEONE's pet feature, and they will think you are cruel and uncaring, and say nasty things about you.
Sigh.
Sometimes the decisions are REALLY hard, like making head to head modem play suffer to enable persistant internet servers.
-----------------------------------------
John Carmack's .plan for Jul 11, 1997
-----------------------------------------
Zoid commented that my last .plan update sounded like Fred Brooks "The Mythical Man-Month". He is certainly correct.
When I read TMMM two years ago, I was stunned by how true and relevent it was. I have something of a prejudice against older computer books - I think "If its more than a five years old, it can't be very relevent" (sure, thats not too rational, but what prejudice is?).
Then I go and read this book that is TWENTY YEARS old, that talks about experience gained IN THE SIXTIES, and I find it mirroring (and often crystalizing) my thoughts on development as my experiences have taught me.
It even got me fired up about documenting my work. For about a day :)
I had to fly out to CA for biz on thursday, so I decided to grab and re-read TMMM on the plane.
It was just as good the second time through, and two more years of development under my belt hasn't changed any of my opinions about the contents.
If you program (or even work around software development), you should read this book.
-----------------------------------------
John Carmack's .plan for Jul 25, 1997
-----------------------------------------
Id Software went to the drag strip today.
The 100 degree heat was pretty opressive, and my NOS regulator wasn't working, but a good time was had by all.
I made six runs in the 126 to 133 mph range and didn't even burn a spark plug, which is a nice change from a couple road track events I have been to.
Best times for everyone:
[code]
Bob Norwood's PCA race car: 10.9 / 133 mph (slicks)
My turbo testarossa 12.1 / 132
Adrian's viper 13.5 / 105
Todd's 'vette 13.9 / 101
Tim's porsche 14.3 / 96
Bear's supra: 14.4 / 96
Cash's M3 15.2 / 94
[/code]
My TR is never going to be a good drag car (>4000 lbs!), but when we go back on a cool day this fall and I get my NOS running, it should be good for over 140 in the quarter. 50 mph to 200 mph is it's real sweet spot.
I think Bear is heading for the chip dealer so he can get ahead of Tim :)
-----------------------------------------
John Carmack's .plan for Jul 30, 1997
-----------------------------------------
quake2 +set maxclients 200
:)
The stage is set for ultra-large servers. Imagine everyone at QuakeCon in one gigantic level! A single T1 could run 80 internet players if it wasn't doing anything else, a switched ethernet should be able to run as many as we are ever likely to have together in one place.
There will be a number of issues that will need to be resolved when this becomes a reality, but the fundamentals are there.
There will probably be issues with UDP packet dropping at the ethernet card level that will need to be worked around with a seperate qued thread.
Quake 2 isn't as cpu intensive as QuakeWorld, but I'm not sure even a Pentium-II 300 could run 200 users. An alpha 21264 could certainly deal with it, though.
The new .bsp format has greatly increased size limits, but you could still make a map that hits them. The first one to be hit will probably be 64k brush sides. Ten thousand brushes can make a really big level if you don't make it incredibly detailed. Loading a monster map like that will probably take over a minute, and require 32+ megs of ram.
I should probably make an option for death messages to only be multicast to people that are in the potentially hearable set, otherwise death messages would dominate the bandwidth.
Everyone should start thinking about interesting rules for huge games. A QuakeArmies dll has immense potential. Enemy lines, conquering teritory, multiple clan bases, etc.
Cooperating servers will be possible with modified dlls, but I probably won't include any specific code for it in the default game.dll.
-----------------------------------------
John Carmack's .plan for Aug 05, 1997
-----------------------------------------
At siggraph
* fix qe4 autosave
* merged qlumpy into qdata, save seperate files
* changed quaked to use texture directories
+ fix leaktest option
+ show texture directory on inspector window
+ show full texture name somewhere when clicked on
+ texture info overrides
remap maps to share common textures?
-----------------------------------------
John Carmack's .plan for Aug 06, 1997
-----------------------------------------
* qe4 texture directories
* fixed vid_restart
* hacked alpha colors for cards without src*dst
* fixed qdata vc compiler bug in arg parsing
* qe4 surface inspector
-----------------------------------------
John Carmack's .plan for Aug 07, 1997
-----------------------------------------
+ add animation frames to bsp file texinfos
- make bmodel frames just add to texinfo?
- should msurface flags hold the texinfo flags?
+ make window content implicit if any surfaces are trans
+ CONTENTS_ORIGIN flag
+ nodetail bsp
+ select face option in qe4
+ use monsterclips!
+ gl fullbright textures are still 2x brightness
moveable alpha surfaces
merge find texture dialog into surface inspector
fix qdata unix directory stuff
get rid of mod->skins, use mod->images
-----------------------------------------
John Carmack's .plan for Aug 08, 1997
-----------------------------------------
* added origin brush support to old bsp for raven
+ add edge planes for brush hulls
- rate is broken - inventory fix
-----------------------------------------
John Carmack's .plan for Aug 09, 1997
-----------------------------------------
* combined bsp tools into a single vc project
* new texture animation solution
* make any com_error drop the loading plaque
* tools and quake2 work with new bsp format
+ combine project files of bsp tools
+ anything translucent is automatically a detail contents
- duplicate texinfo for animations?
+ store out contents from trace!
+ arbitrary visleafs mappings
+ scanmaps option for pak file building of textures
+ delta lightstyle controls from server
+ max moveleafs problem
+ make r_dowarp a server passed variable?
+ why is hunk_begin different in software?
don't forget to set SURF_NOSUBDIV on warps and sky!
compress ff in visdata as well as 0?
trinity idea: model light haze around every emiter
trinity idea: allways model volumetric lights by rendering back sides
CONTENTS_VOLUME
do a wavy specular water novelty
allow arbitrary chained lightmaps on a surface?
game.dll controlable particles
get rid of SURF_PLANEBACK
player sounds when moving? (breathing / footsteps / hitting walls)
rename .bsp to .bs2 ?
high frame rate run turn chunkiness
-----------------------------------------
John Carmack's .plan for Aug 10, 1997
-----------------------------------------
I went to siggraph last monday to give a talk about realtime graphics for entertainment.
The only real reason I agreed to the talk (I have turned down all other offers in the past) was because Shigeru Miyamoto was supposed to be on the panel representing console software. Id software was really conceived when me, Tom, and Romero made a Super Mario 3 clone after I figured out how to do smooth scrolling EGA games. We actually sent it to nintendo to see if they wanted to publish a PC game, but the interest wasn't there. We wound up doing the Commander Keen games for Apogee instead, and the rest is history.
I was looking forward to meeting Mr. Miyamoto, but he wound up canceling at the last minute. :(
Oh well. I hope everyone that went enjoyed my talk. All the other speakers had powerpoint presentations and detailed discussion plans, but I just rambled for an hour..
I notced that there was a report about my discussion of model level of detail that was in error. I have an experimental harness, an algorithm, and a data structure for doing progressive mesh style LOD rendereing in the quake engine, but I suspect it won't make it into the production Quake 2. Other things are higher priority for us. I may assist some of the quake licensees if they want to pursue it later.
-
A couple data / feature changes going into the latest (and I hope final) revision of the Quake bsp file format:
Back in my update a month ago where I discussed losing automatic frame animation in models to clean up the format and logic, I mentioned that I still supported automatic texture animation.
Not anymore. There were several obnoxious internal details to dealing with it, especially now with textures outside the bsp file, so I changed the aproach.
When a texture is grabbed, you can now specify another texture name as the next animation in a chain. Much better than the implicit-by-name specification from Quake1.
No animation is automatic now. A bmodel's frame number determines how far along the animation chain to go to find the frame. Textures without animation chains just stay in the original frame.
There is a slight cost in network traffic required to update frame numbers on otherwise unmoving objects, but due to the QuakeWorld style delta compression it is still less than a Quake 1 scene with no motion at all.
The benefit, aside from internal code cleanliness, is that a game can precisely control any sequence of animation on a surface. You could have cycles that go forward and backwards through a sequence, you could make slide projectors that only change on specific inputs, etc.
You could not independantly animate two sides of a bmodel that were not syncronized with the same number of frames, but you could allways split it into multiple models if your really needed to.
Everything is simple when its done, but I actually agonized over animation specification for HOURS yesterday..
The last significant thing that I am working on in the map format is leaf clustering for vis operations. You can specify some map brushes as "detail" brushes, and others as "structural" brushes. The BSP and portal list is built for just the structural brushes, then the detail brushes are filtered in later.
This saves a bit of space, but is primarily for allowing complex levels to vis in a reasonable amount of time. The vis operation is very sensitive to complexity in open areas, and usually has an exponentially bad falloff time. Most of the complexity is in the form of small brushes that never really occlude anything. A box room with ten torch holders on the walls would consist of several dozen mostly open leafs. If the torch holders were made detail brushes, the room would just be a single leaf.
A detail / structural seperation is also, I believe, key to making a portal renderer workable. I had a version of Quake that used portals at the convex volume level, and the performance characteristics had considerably worse-than-linear falloff with complexity. By reducing the leaf count considerably, it probably becomes very workable. I will certainly be reevaluating it for trinity.
* trans33, trans66, flow flags in gl
* damped warp modulation in gl
* ref_soft running with new data
+ shots are exploding on the sky again
+ auto set window contents if translucent
+ don't set qe4 texture unless notexture
+ try new console background
+ finish animation cycling
detail brushes could be extended to be destroyable
new texture specification by three points?
check -tmpin -tmpout in bsp utils
rename texinfo to surfinfo?
pitch change during jumping
minimized window notification when a new client joins?
should origin brushes be included in bsp file for completeness?
use nodraw flag
pitch change when ducking
qrad light bleeds
-----------------------------------------
John Carmack's .plan for Aug 11, 1997
-----------------------------------------
* don't set qe4 texture unless notexture
* don't set qe4 texture on cancel unless changed
* grabbed new menu and console
* invert mouse off in default.cfg
* all software flags
* mist contents
+ imagelist command in software
trinity: save out projection outlines from editor for textures
add a 5th control axis (and 6th?) for spaceorb ducking
gl: don't keep lightmap blocks around in main memory?
entities not visible (or only visible) to owners
look in direction other than motion for hmd
quake as root directory problem
dir command
software surface / edge allocation issues
-----------------------------------------
John Carmack's .plan for Aug 12, 1997
-----------------------------------------
* qe4 project on command line
* qe4 rshcmd replacement
* qe4 select face
* qe4 avoid multiple autosaves
* qe4 region selected brushes
* bindlist command
* imagelist command in ref_soft
+ leaktest
+ load game.dll from gamedir
pendulum motion
no jump on lava floor?
-game
16 bit wall textures
-----------------------------------------
John Carmack's .plan for Aug 13, 1997
-----------------------------------------
* cls.fixedimage support
* no frame before cinematic fix
* menu during cinematic fix
+ ingame cinematic state
+ indemo cinematic state
- move fraglogfile into game dll
+ layout language beyond simple centerprint
+ killserver needs to kill demos as well
+ must kill cinematic after menu, or restart palette
+ disconnected can be either at a console or running the demo + intro cinematic
needs to be part of the game
force nolerp lag?
put ip filtering in game dll
handle localmodels explicitly, rather than as *num
don't send heartbeats if not running a network game?
move viewmodel for all accelerations, including jumping and landing
fade out centerprints
design quit screen to allow addons to get credits
be consistant with window title bars
mp3 audio
qe4: downsample option, nomipmap option
-----------------------------------------
John Carmack's .plan for Aug 14, 1997
-----------------------------------------
* qe4 project dialog fix
* intermission spots and movement
* hud transfer framework
+ micro levels that just play cinematics?
+ BUTTON_ANY option
remove oldorigin
use static arrays for map elements in renderers?
unit level statistics
-----------------------------------------
John Carmack's .plan for Aug 15, 1997
-----------------------------------------
* smart background clear
* worked around 100% viewsize floating point bug
* increased base surface cache size
* unified server commands and prog commands
* fixed same level reload bug in ref_soft
+ are lightmaps allways being saved, even if all black?
+ is notify box used for anything?
+ toggleconsole when connected to a net game
+ server needs to be able to send staticpics
+ draw to front buffer without swapping option
- can game.dll register commands?
+ direct sound and keyboard restart so ref can destroy window
+ loading plaque on local system doesn't wait for hud erase
frame flicker option for evaluating missed VBL?
way to add remote commands on client side by scripts?
check client entering during intermission
moveable transparant bmodels
use sneaking in shadows to let players get a good look at more monsters
translate cinematic to greyscale instead of blanking?
remove zombietime?
are userinfo updates getting sent?
-----------------------------------------
John Carmack's .plan for Aug 16, 1997
-----------------------------------------
download static screen images?
+ how to change semi-protected variables without killing server?
+ how do demo servers progress to the next server?
+ how does the client distinguish between a demo server? parm to map command?
+ demo servers have special console behavior and don't warn on game restart
+ do not allow remote connects to a demo server
+ no loading plaque if fullcon
+ cinematic trailing pic for victory
+ demo view angles?
+ text before next level after completed
+ replace draw_beginrawscene with setpalette?
+ keys should go to game when running cinematic, not console
+ does the console key serve as a skip-demo key on real servers?
+ need to flag unit ends for stats, vs simple transfers
+ pause demos and cinematics while menu is up
visible que on players when typing and when lagged?
make sure there is never a server running if client is fullcon
must force full graphics reload on game change
don't require full precache of all weapons?
demo servers won't work for remote connections, because packets can be dropped
prevent map / demomap changes without killserver
map demo1.dem during game messes up
victory freeze
-----------------------------------------
John Carmack's .plan for Aug 17, 1997
-----------------------------------------
* demo angles
* fixed initial lightmap cache value
* disconnect now does an ERR_DROP to kill server as well
* button_any support
* bad fov problem
aliaslist
never nextserver on finale
blaster autorepeat problem
end cinematic loading flicker
-----------------------------------------
John Carmack's .plan for Aug 18, 1997
-----------------------------------------
I get asked about the DOOM source code every once in a while, so here is a full status update:
The Wolfenstein code wasn't much of a service to release - it was 16 bit dos code, and there wasn't much you could do with it. Hell, I don't think it even compiled as released.
The DOOM code should be a lot more interesting. It is better written, 32 bit, and portable. There are several interesting projects that immediately present themselves for working with the code. GLDOOM and a packet server based internet DOOM spring to mind. Even a client/server based DOOM server wouldn't be too hard to do.
I originally intended to just dump the code on the net quite some time ago, but Bernd Kreimeier offered to write a book to explain the way the game works. There have been a ton of issues holding it up, but that is still the plan. If things aren't worked out by the end of the year, I will just release things in a raw form, though.
My best case situation would be to release code that cleanly builds for win32 and linux. Bernd is doing some cleanup on the code, and some of the Ritual guys may lend a hand.
One of the big issues is that we used someone else's sound code in dos DOOM (ohmygod was that a big mistake!), so we can't just release the full code directory. We will probably build something off of the quake sound code for the release.
I think I am going to be able to get away with just making all the code public domain. No license, no copyleft, nothing. If you apreciate it, try to get a pirate or two to buy some of our stuff legit..
* lightmap building errors
+ qe4: build in detail mode
+ animating textures
+ no different quantities on items?
+ target_secretcounter
the inherent problems of simplicity by complexity
-----------------------------------------
John Carmack's .plan for Aug 19, 1997
-----------------------------------------
* leaktest
+ min clamp extents
-----------------------------------------
John Carmack's .plan for Aug 23, 1997
-----------------------------------------
* cluster code
- boxcontents?
+ dump rgb lightmaps for software?
+ alias model aspect ratios different in software and gl?
share data between cmodel and ref
triangulate mightsee on vis?
malloc all cmodel arrays?
DONT_PRECACHE flag for player weapons?
make an ERR_DISCONNECT that doesn't print ERROR: ?
don't load entire map twice in cmodel and ref!
show clusterviscount for bsp time optimizations?
server/client communication for skin overrides
-----------------------------------------
John Carmack's .plan for Aug 24, 1997
-----------------------------------------
* qe4 slow startup
* qrad
+ detail clip brushes?
+ extra brush clip planes
change qdata colormap to not use 0 and 255 for win palette
-----------------------------------------
John Carmack's .plan for Aug 25, 1997
-----------------------------------------
I want to apologize for some of the posturing that has taken place in .plan files.
I have asked that attacks on our competition no longer apear in .plan files here. I don't think it is proper or dignified.
If everyone clearly understood that an individual's opinion is only that - the opinion of a single individual, I wouldn't have bothered. Unfortunately, opinions tend to be spread over the entire group, and I am not confortable with how this makes me perceived.
Building up animosity between developers is not a very worthwhile thing.
A little chest-beating doesn't really hurt anything, but putting down other developers has negative consequences.
I think that we have a track record that we can be proud of here at id, but we are far from perfect, and I would prefer to cast no stones.
The user community often exerts a lot of pressure towards confrontation, though. People like to pick a "side", and there are plenty of people interested in fighting over it. There are a lot of people that dislike id software for no reason other than they have chosen another "side". I don't want to encourage that.
Magazine articles are usually the trigger for someone getting upset here. Its annoying to have something you are directly involved in misrepresented in some way for all the world to see. However, I have been misquoted enough by the press to make me assume that many inflamatory comments are taken out of context or otherwise massaged. It makes a good story, after all.
Sure, there ARE developers that really do think they are going to wipe us off the face of the earth with their next product, and don't mind telling everyone all about it. Its always possible. They can give it their best shot, and we'll give it ours. If they do anything better, we'll learn from it.
* fixed water bsp bug! yeah!
* new tools in production
* view pitching with running
* weapon turn lagging
* debug graph
+ screen sizedown is not clearing again
+ animating textures
+ weapon change sounds should be server side
+ QE4: surface inspector apply is slow
qe4: seperate "make current texture" from "make and set all"
currentmodel name problem in gl_model
userinfo changes
-----------------------------------------
John Carmack's .plan for Aug 26, 1997
-----------------------------------------
* debuggraph on top
* better bobtime / bobcycle
* face seperation overrun bug
* fast surface dialog
* show detail on camera title
+ link commands for playing from the cd
qe4: view menu checkboxes are wrong
-----------------------------------------
John Carmack's .plan for Aug 27, 1997
-----------------------------------------
* fixed off-by-one cluster count
* fixed surf/content bit mismatch
* gun bob cycles
* falling pitch change
- make a fat pvs for rendering?
+ trace needs to return contents
+ fix BUTTON_ANY
+ rendering beams
+ delta lightstyle controls from server
+ finish animation cycling
QE4: deleting brushes doesn't count as modified?
initial time on spawn Tent
underwater caustics
make all bobbing effects cvars
title on inspector is broken for textures
moveable alpha surfaces
don't forget to set SURF_NOSUBDIV on warps and sky!
freeze map just sets a HUD of the victory screen
server scoreboard
-----------------------------------------
John Carmack's .plan for Aug 28, 1997
-----------------------------------------
* fixed entity culling on gl
* sorted axial and edge bevels on all brushes
+ entity culling in GL
+ imagelist should have the downsampled sizes
+ software should dump rgb lightmap data
an origin brush will never change a texinfo?
NO! the offsets can change
are brush numbers messed up because of removed brushes?
plat push into floor
use textureisresident in imagelist?
load mip levels seperately
duplicate planes
make set detail not work on entities
trinity: pivot feet! general atmospherics!
ray trace: texture+s/t for each sample, hardware reconstructs
walk up stairs by slope hitches up
animating textures
QE4: use gentextures
QE4: flush all textures option
-----------------------------------------
John Carmack's .plan for Aug 29, 1997
-----------------------------------------
sick
-----------------------------------------
John Carmack's .plan for Aug 30, 1997
-----------------------------------------
* changed snapnormal
* fixed BUTTON_ANY
* unix makefile
* pic server
* runcinematic call
* console over cinematic fix
+ console key during game cinematics
+ version number for quake 2?
+ cinematic set palette needs to clear screen
use cluster level leafs for sound radiosity
jittered texel centers?
trinity: continuous textures, surface cache on all
make net, pause, and drawcenterstring HUDs
-----------------------------------------
John Carmack's .plan for Aug 31, 1997
-----------------------------------------
* fixed trigger_always
* game dll by search path
* cinematic NULL bug
* help computer
+ get rid of datablocks?
+ dll init must clear the persistant data
- savegame needs to save game.dll name
+ save directory?
+ put pcx_t into qfiles.h?
+ unify all hud work into g_hud.c
set command with flags?
should "path" be renamed to pathlist?
trigger_allways should be fixed size
gameversion
somehow don't resend big deltas (scoreboard hud) until ack?
client side feedback
glgentextures
-----------------------------------------
John Carmack's .plan for Sep 01, 1997
-----------------------------------------
* QE4: bug with texture attributes on non-visible surfaces
* stack bug for initial light maps
+ pink lightmaps?
+ alt-tab should only minimize if full screen
+ version as command and var?
splashing sound when swimming at surface?
brains make view roll around
wasted polygons outside maps
vis decompression overruns?
INVISIBLE_TO_OWNER INVISIBLE_TO_OTHER
make * model names visible
GL_MipMap overwriting?
trinity: proper biped walking
increase range of mouse slider
gun shock based on damage source
-----------------------------------------
John Carmack's .plan for Sep 02, 1997
-----------------------------------------
* serveronly semeaphor close check
* error during initialization messagebox check
* software rendering default wall image
* unify slidestepmove
+ progress bar
+ roll angles are getting set on rotating models
+ blinking lights on bonus items
+ alpha test fix
bug with loading a new map after an errored out map
qe4: turn region off for new map
gun should show vertical acceleration on lifts
view angle turning based on impacts
muzzle flashes
trinity: fully compressed textures need to compress the mip levels as well, instead of generating them from the most detailed form.
trinity: different packing options to layout all the texture blocks. Square, thin, individual, etc.
trinity: investigate performance of background disk paging and clustering of texture blocks into disk allocation units
trinity: texture connectivity graph for prepaging?
trinity: speculatively upload things that might be seen next frame to balance uploading?
max upload, use lower mip levels if needed?
get rid of all the gl lightmaps in main memory!
allow jump up off bottom of water floor to give more velocity
merge net_udp and net_wins
slippery slopes
learned something: upload mip levels backwards
-----------------------------------------
John Carmack's .plan for Sep 03, 1997
-----------------------------------------
* fixed scalloc size 0
* box on planeside fix
* remove 0 and 255 colormap references
* don't allocate texinfos for empty texture names
* fix the initial teleport spawn timing bug
* sinking into plats
* exaggerate stepping when crouched?
+ allow pics off screen (status bar off bottom of screen)
+ check control configuration
+ make moveup jump
+ animating textures
+ base window not very noticable
+ any flowing?
+ fix deltaangle hack in putclientinserver
+ move copytooldorg to prepworldframe
+ demos
+ r_speeds include particles
+ escape should pause demos
reduce acceleration on low grav levels
moving translucent objects
timedemo leaves console in attract key mode
underwater speed is too great
flex legs intentionally on plats?
forward when facing an obstruction directly should not slide
clip brush fragments in base
exit button clip stuck problem
no such oldframe -1
melee attacks out of range
rename g_client to p_client
merge cl_fx and cl_tent
deal with oldorgs better
software menu black screen flickers
rethink scrap allocation
crouchwalk up slopes is fucked up (stair uping)
r_stats include bind counts
change lightmaps into images
make gl_bind() take an image, so it can reference sizes
script parsing should take /* */ and line continuation
reduce skies?
3dfx opengl: detect thrashing and split the cache?
demos don't read from pak files??????
muzzle flashes
forward / backwards airlocks
better button representations
brighter primary colors
-----------------------------------------
John Carmack's .plan for Sep 04, 1997
-----------------------------------------
* mcd alphatest workaround
* gl_finish
* gl_dynamic
* fixed crash without basedir
+ savegame harness
+ loading plaque
+ proper alias bbox in gl
+ light feedback for server
+ 3dfx vid_mode problem
+ remove MAX_SCRAPS
- arbitrary skin support needed for power armor
sort entities by texture
segment skies up more to get better caching?
circle monsters pvs
are gl sprites double bright?
-----------------------------------------
John Carmack's .plan for Sep 05, 1997
-----------------------------------------
* fixed all angle clamping issues
* allow look up / down to exactly 90 degrees
red numbers on status bar
-----------------------------------------
John Carmack's .plan for Sep 07, 1997
-----------------------------------------
* timedemo attract flag bug
* multiple model entities
* 8/16 bit updates
* merged delta with baseline
+ destroy windows on each ref start
+ server time clamping issues
+ client light level different between refs
+ remove version command
+ animating textures
+ run from cd option
+ toast mergedemos
flags not used in entity-state?
airborne frames for everything?
don't send player entity to owner in most cases
client quit dropping prints extra messages?
trinity: track and field style extra fast running?
-----------------------------------------
John Carmack's .plan for Sep 08, 1997
-----------------------------------------
* fixed time clamping issue
* removed win32isms from snd_mix.c and snd_dma.c
* optimized dsound locks
+ scale texture stats by texture size
+ not autodetecting PII for mono lightmaps?
+ put swaps from ref into qshared?
+ walk backwards when looking straight up?
seperate #define for asm code?
swim up with jump key
window close box
reload textures only on context recreation
do a stereo gl implementation on intergraph?
item using...
-----------------------------------------
John Carmack's .plan for Sep 09, 1997
-----------------------------------------
* freed textures on gl shutdown
* fixed pitch bounds check merge bug
* cleanup sound code
* cut default maxclients
* cut update_backup
+ soft fullscreen failure on ingr
+ error not closing window
+ cds window set on top of taskbar
+ fullscreen in mode 0, set mode 1 = crash
+ alt stuck down after alt-tab
fix vis expansion problem
trinity: shimmering heat atmospheric effects
need a remove command builtin for game logic
get all texture extension numbers into gl_image
vis decompression overrun
window doesn't offset in non-fullscreen modes
-----------------------------------------
John Carmack's .plan for Sep 10, 1997
-----------------------------------------
* don't precache player model in single player games
* dynamically change maxplayers
+ move null files into a seperate directory?
+ alt sticking
+ maxmoveleafs error
teleport in flash is still wrong on second level
stuck on wall with low grav jumping
allow minimize?
win95 memory paging
still have some tjunctions
-----------------------------------------
John Carmack's .plan for Sep 11, 1997
-----------------------------------------
* cddir
+ must save status upon entering a level if it was a new spawn
- each map has a unit number and a level number
is changing skill/etc going to be a problem while demos are running?
demos in pak file
don't use virtual alloc!
-----------------------------------------
John Carmack's .plan for Sep 12, 1997
-----------------------------------------
+ tag_game, tag_level
+ new game must clear
- need to save game state at last level entered as well as exact current
+ save level on exit..
+ spawnpoint to game dll wrong?
+ collapse sv_phys into something else?
+ skip all pixelformat stuff on minidrivers?
get rid of Com_SetServerState() ?
ERR_DROP?
status command that prints out server and client state?
don't allow anything but CRC checked pak file access in a demo
client userinfo change updates
worry about cvar info string lengths
make sure IP is visible to game
track deaths as well as fregs?
directmouse?
view shaking from explosions?
-----------------------------------------
John Carmack's .plan for Sep 13, 1997
-----------------------------------------
+ skill levels need to be archived with server state!
+ angle clamp on server is broke again
+ don't shrink status bar with window
+ make sure all char * in edicts are copies, not just references
+ difference between reentering a level and reloading it
check all savegame files for disk space errors.
current is automatically updated whenever a level is exited
archive the level being exited if not leaving the unit
save the map to be entered, SKIPPING ANY CINEMATICS!
end of game will not have a final map, so don't save
savegame does NOT update current, the level archive and server
is written directly to the new directory
new game
single player game
on death + press, bring up loadgame screen
deathmatch
on death + press, respawn
coop
on death + press, respawn
need to have the game start up without TCP/IP unless asked for
dir command with sys_find*
ping equalization?
set userinfo->ip on each server connect
high quality / low quality sound option in menu, create a special sample for
testing
fix svs / sv to be more game/level oriented
make coop games allways four player?
wav lag seems worse
-----------------------------------------
John Carmack's .plan for Sep 14, 1997
-----------------------------------------
* qdata grab alias numeric suffix
* menu architecture
+ make a portal entity
+ connect doors to portal entities
+ treat portal contents like windows
+ flood fill leafs, but stop at portals to count areas
+ if actual leaf with portal contents should chose any area next + to it
+ each portal brush should have exactly two areas bordering it
+ server sends over a bit vector of areas visible to player
+ use area visibility as fast reject for line testing?
+ should portal entities remain seperate, or just add a portal + field
+ to doors?
+ builtin: SetPortalState (int pnum, qboolean open);
+ portals MUST go in the structural bsp!
+ each leaf has an area field
+ each portal has two areas it connects
+ all other data can be derived
+ areas have the list of portals
+ area * area * 2^portals == too large!
+ must do dynamic flood fill
+ most portals will be closed, so flood fill is fast
+ game pause
+ no status bar after death
allow higher precision bmodel angle specification?
put v_idle back in?
super crouch jump?
establish a client connection at startup to avoid localconnects?
more bright areas in the game for contrast?
throbbing health status pic
weapon cycle command
bigger font?
allways have visible blood particles by face when hit?
bounce health status around when hit?
radius of alias model dlighting is greater than surface dlighting
trinity: use mouse curssors for ui stuff?
menu_move, menu_down, menu_up, menu_change, menu_slide
trinity: software trilinear with second pass? only works if vertex lighting
-----------------------------------------
John Carmack's .plan for Sep 15, 1997
-----------------------------------------
+ cinematic paking!
+ r_dspeeds should include translucent time
+ alt key stuck donw after alt-enter
+ bonus flashes
texture releasing from maps isn't uniqued
scissor triangles
faster z clip
make autoexec.cfg work differently because of demos
-----------------------------------------
John Carmack's .plan for Sep 16, 1997
-----------------------------------------
* finished box sweeping code
* fix the automenu key problem on bad cinematic
* blinking black screen palette set issues
* send dowarp over from server
* fixed color 0 grabbing
* bonus flashes
* Q_fabs
+ dedicated server
drop stair climb in air height
-----------------------------------------
John Carmack's .plan for Sep 17, 1997
-----------------------------------------
* fixed qe4 texturepath bug
* qe4: show in use textures even when showing a wad
* utils: fixed gamedir to allow nested quake2 directories
* moved env out of gfx, so gfx is all source files
malloc tags for game and level
clone detail brushes should remove detail flag
make timedemo a server connect thing
ktest.reg bad model
-----------------------------------------
John Carmack's .plan for Sep 18, 1997
-----------------------------------------
* finished code dump
* dedicated server
* removed all dash parms
* texture paging research
+ examine ambient sounds
+ key clear events doesn't clear everything
sound streaming
bsp hint brushes (SURF_NODRAW?)
autoexec.cfg
ip cvar for multihomed servers
ip userinfo for clients
report dash parms on cmd line?
menu on top of cinematic leaves crap
color 0 is still broken on NT
allow clients to connect to the server even if it is not
running a level?
local client is allways connected
clients are only kicked when the entire server is shut down
or they connect to a remote server
-----------------------------------------
John Carmack's .plan for Sep 19, 1997
-----------------------------------------
* basedir / cddir exec problem
* moved edict allocation into game
+ only change yaw on riding bmodels
+ city3 software crash
+ odd pause before connecting to map
+ !!!SV_PointContents needs to check entities!!!
+ areaportal numbers
+ move spawn/free into game logic?
are sound starts lagged by 0.1? only lag offsets?
get all cvars for game into one place
send objects only to owner or vs versa flag
loading plaque from post cinematic "you win!" screen
QE4: fix the idle redraw problem
vis decompress overrun
get rid of zombie state?
-----------------------------------------
John Carmack's .plan for Sep 20, 1997
-----------------------------------------
dead air conditioning
-----------------------------------------
John Carmack's .plan for Sep 21, 1997
-----------------------------------------
* areaportals!!!
* model contents for moving water
+ use different decompression buffers for pvs and phs?
+ fix headnode issues
get rid of fat pvs completely?
-----------------------------------------
John Carmack's .plan for Sep 22, 1997
-----------------------------------------
* fullsend headnode done properly
+ animating textures
+ must check all cluster areas for doors
-1 cluster issues?
more dramatic railgun spray
check all the trace flags to see if they are still needed
-----------------------------------------
John Carmack's .plan for Sep 23, 1997
-----------------------------------------
* fixed bmodel cluster overload
* double areas for doors
* drawworld 0
* bmodel sound origins
+ rename "globals" in game to ge?
+ remove limits on max entities in packet?
+ better way of tracking static sounds, so they can be turned off?
+ object looping sounds?
+ machine sounds in fact2
+ fix look up / run back bug
+ add more packet buffers to avoid "dumped packet" warnings?
+ dll basing information for function pointers
finish status bar and inventory
use areas for multicast
hint brushes
eliminate baselines by allways tracking client's knowledge of all ents?
qdata model frame auto-number issues
snap stuck view when dead?
set_ex command to set info status
game dll version number?
-----------------------------------------
John Carmack's .plan for Sep 24, 1997
-----------------------------------------
A significant new feature for map development sneaked into Quake 2 this week.
It has always been a problem with Quake that putting a door in front of a complex area didn't make the scene run any faster, unlike DOOM. In glquake, it actually made it significantly slower as you aproached the door, due to overdraw.
There was also the related problem that monsters heard sounds through doors even if they were closed.
This was because the primary culling mechanism used by Quake is the PVS - the Potentially Visible Set. It only knew about anything that you could POTENTIALLY see from your current (rough) position. If a door might open, the PVS would allways contain everything that you could see even if the door was currently closed.
Quake 2 now has a way to allow you to lop off large amounts of the map irrespective of the PVS.
A map is now divided into "areas" by "areaportal" entities, usually in door frames.
If the area behind a door is not reachable by any open areaportals, then nothing from that area will be visible or hearable. This helps both rendering speed and network bandwidth. It also give the level designer an easy "band-aid" when they have designed an area that is too slow.
Note that the area-reachable test is strictly a topological flood fill, so if there is ANY route to the other side of a door open, you will still be processing the area behind the door, even if there is no real way you could see through the available route.
If your level has a reasonable number of doors, it will often run at a fair speed without any PVS information at all.
To use this feature, you create a thin "func_areaportal" entity that hides completely inside the door, then target the door at it. Qbsp3 does a bunch of work behind your back that you really don't want to know about. Doors have special logic in the game to open or close the areaportal at the apropriate time.
I chose not to make it an automatic feature of doors for a few reasons:
1) Teamed double or quad doors would not create a single portal across the entire doorway.
2) The areaportal entity can also be used for things like exploding walls. You can even put one just around a corner and trigger it with a field, but it is usually better to just let the PVS take care of corner bends.
3) Complex doors would have created complex (but invisible) area portal brushes, which would have messed up the bsp a bit.
I think this was the very last data file change for quake II, so here is the current external files header for the curious: (4 character tabs)
[code]
//
// qfiles.h: quake file formats
// This file must be identical in the quake and utils directories
//
/*
========================================================================
.MD2 triangle model file format
========================================================================
*/
#define IDALIASHEADER (('2'<<24)+('P'<<16)+('D'<<8)+'I')
#define ALIAS_VERSION 8
#define MAX_TRIANGLES 4096
#define MAX_VERTS 2048
#define MAX_FRAMES 512
#define MAX_MD2SKINS 32
#define MAX_SKINNAME 64
typedef struct
{
short s;
short t;
} dstvert_t;
typedef struct
{
short index_xyz[3];
short index_st[3];
} dtriangle_t;
typedef struct
{
byte v[3]; // scaled byte to fit in frame mins/maxs
byte lightnormalindex;
} dtrivertx_t;
#define DTRIVERTX_V0 0
#define DTRIVERTX_V1 1
#define DTRIVERTX_V2 2
#define DTRIVERTX_LNI 3
#define DTRIVERTX_SIZE 4
typedef struct
{
float scale[3]; // multiply byte verts by this
float translate[3]; // then add this
char name[16]; // frame name from grabbing
dtrivertx_t verts[1]; // variable sized
} daliasframe_t;
// the glcmd format:
// a positive integer starts a tristrip command, followed by that many
// vertex structures.
// a negative integer starts a trifan command, followed by -x vertexes
// a zero indicates the end of the command list.
// a vertex consists of a floating point s, a floating point t,
// and an integer vertex index.
typedef struct
{
int ident;
int version;
int skinwidth;
int skinheight;
int framesize; // byte size of each frame
int num_skins;
int num_xyz;
int num_st; // greater than num_xyz for seams
int num_tris;
int num_glcmds; // dwords in strip/fan command list
int num_frames;
int ofs_skins; // each skin is a MAX_SKINNAME string
int ofs_st; // byte offset from start for stverts
int ofs_tris; // offset for dtriangles
int ofs_frames; // offset for first frame
int ofs_glcmds;
int ofs_end; // end of file
} dmdl_t;
/*
========================================================================
.SP2 sprite file format
========================================================================
*/
#define IDSPRITEHEADER (('2'<<24)+('S'<<16)+('D'<<8)+'I')
// little-endian "IDS2"
#define SPRITE_VERSION 2
typedef struct
{
int width, height;
int origin_x, origin_y; // raster coordinates inside pic
char name[MAX_SKINNAME]; // name of pcx file
} dsprframe_t;
typedef struct {
int ident;
int version;
int numframes;
dsprframe_t frames[1]; // variable sized
} dsprite_t;
/*
==============================================================================
.WAL texture file format
==============================================================================
*/
#define MIPLEVELS 4
typedef struct miptex_s
{
char name[32];
unsigned width, height;
unsigned offsets[MIPLEVELS]; // four mip maps stored
char animname[32]; // next frame in animation chain
int flags;
int contents;
int value;
} miptex_t;
/*
==============================================================================
.BSP file format
==============================================================================
*/
#define IDBSPHEADER (('P'<<24)+('S'<<16)+('B'<<8)+'I')
// little-endian "IBSP"
#define BSPVERSION 38
// upper design bounds
// leaffaces, leafbrushes, planes, and verts are still bounded by
// 16 bit short limits
#define MAX_MAP_MODELS 1024
#define MAX_MAP_BRUSHES 8192
#define MAX_MAP_ENTITIES 2048
#define MAX_MAP_ENTSTRING 0x20000
#define MAX_MAP_TEXINFO 8192
#define MAX_MAP_AREAS 256
#define MAX_MAP_AREAPORTALS 1024
#define MAX_MAP_PLANES 65536
#define MAX_MAP_NODES 65536
#define MAX_MAP_BRUSHSIDES 65536
#define MAX_MAP_LEAFS 65536
#define MAX_MAP_VERTS 65536
#define MAX_MAP_FACES 65536
#define MAX_MAP_LEAFFACES 65536
#define MAX_MAP_LEAFBRUSHES 65536
#define MAX_MAP_PORTALS 65536
#define MAX_MAP_EDGES 128000
#define MAX_MAP_SURFEDGES 256000
#define MAX_MAP_LIGHTING 0x200000
#define MAX_MAP_VISIBILITY 0x100000
// key / value pair sizes
#define MAX_KEY 32
#define MAX_VALUE 1024
//=============================================================================
typedef struct
{
int fileofs, filelen;
} lump_t;
#define LUMP_ENTITIES 0
#define LUMP_PLANES 1
#define LUMP_VERTEXES 2
#define LUMP_VISIBILITY 3
#define LUMP_NODES 4
#define LUMP_TEXINFO 5
#define LUMP_FACES 6
#define LUMP_LIGHTING 7
#define LUMP_LEAFS 8
#define LUMP_LEAFFACES 9
#define LUMP_LEAFBRUSHES 10
#define LUMP_EDGES 11
#define LUMP_SURFEDGES 12
#define LUMP_MODELS 13
#define LUMP_BRUSHES 14
#define LUMP_BRUSHSIDES 15
#define LUMP_POP 16
#define LUMP_AREAS 17
#define LUMP_AREAPORTALS 18
#define HEADER_LUMPS 19
typedef struct
{
int ident;
int version;
lump_t lumps[HEADER_LUMPS];
} dheader_t;
typedef struct
{
float mins[3], maxs[3];
float origin[3]; // for sounds or lights
int headnode;
int firstface, numfaces; // submodels just draw faces
// without walking the bsp tree
} dmodel_t;
typedef struct
{
float point[3];
} dvertex_t;
// 0-2 are axial planes
#define PLANE_X 0
#define PLANE_Y 1
#define PLANE_Z 2
// 3-5 are non-axial planes snapped to the nearest
#define PLANE_ANYX 3
#define PLANE_ANYY 4
#define PLANE_ANYZ 5
// planes (x&~1) and (x&~1)+1 are allways opposites
typedef struct
{
float normal[3];
float dist;
int type; // PLANE_X - PLANE_ANYZ ?remove? trivial to regenerate
} dplane_t;
// contents flags are seperate bits
// a given brush can contribute multiple content bits
// multiple brushes can be in a single leaf
// lower bits are stronger, and will eat weaker brushes completely
#define CONTENTS_SOLID 1 // an eye is never valid in a solid
#define CONTENTS_WINDOW 2 // translucent, but not watery
#define CONTENTS_AUX 4
#define CONTENTS_LAVA 8
#define CONTENTS_SLIME 16
#define CONTENTS_WATER 32
#define CONTENTS_MIST 64
#define LAST_VISIBLE_CONTENTS 64
// remaining contents are non-visible, and don't eat brushes
#define CONTENTS_AREAPORTAL 0x8000
#define CONTENTS_PLAYERCLIP 0x10000
#define CONTENTS_MONSTERCLIP 0x20000
// currents can be added to any other contents, and may be mixed
#define CONTENTS_CURRENT_0 0x40000
#define CONTENTS_CURRENT_90 0x80000
#define CONTENTS_CURRENT_180 0x100000
#define CONTENTS_CURRENT_270 0x200000
#define CONTENTS_CURRENT_UP 0x400000
#define CONTENTS_CURRENT_DOWN 0x800000
#define CONTENTS_ORIGIN 0x1000000 // removed before bsping an entity
#define CONTENTS_MONSTER 0x2000000 // should never be on a brush, only in game
#define CONTENTS_DEADMONSTER 0x4000000
#define CONTENTS_DETAIL 0x8000000 // brushes to be added after vis leafs
#define CONTENTS_TRANSLUCENT 0x10000000 // auto set if any surface has trans
#define CONTENTS_LADDER 0x20000000
typedef struct
{
int planenum;
int children[2]; // negative numbers are -(leafs+1), not nodes
short mins[3]; // for frustom culling
short maxs[3];
unsigned short firstface;
unsigned short numfaces; // counting both sides
} dnode_t;
typedef struct texinfo_s
{
float vecs[2][4]; // [s/t][xyz offset]
int flags; // miptex flags + overrides
int value; // light emission, etc
char texture[32]; // texture name (textures/*.wal)
int nexttexinfo; // for animations, -1 = end of chain
} texinfo_t;
#define SURF_LIGHT 0x1 // value will hold the light strength
#define SURF_SLICK 0x2 // effects game physics
#define SURF_SKY 0x4 // don't draw, but add to skybox
#define SURF_WARP 0x8 // turbulent water warp
#define SURF_TRANS33 0x10
#define SURF_TRANS66 0x20
#define SURF_FLOWING 0x40 // scroll towards angle
#define SURF_NODRAW 0x80 // don't bother referencing the texture
// note that edge 0 is never used, because negative edge nums are used for
// counterclockwise use of the edge in a face
typedef struct
{
unsigned short v[2]; // vertex numbers
} dedge_t;
#define MAXLIGHTMAPS 4
typedef struct
{
unsigned short planenum;
short side;
int firstedge; // we must support > 64k edges
short numedges;
short texinfo;
// lighting info
byte styles[MAXLIGHTMAPS];
int lightofs; // start of [numstyles*surfsize] samples
} dface_t;
typedef struct
{
int contents; // OR of all brushes (not needed?)
short cluster;
short area;
short mins[3]; // for frustum culling
short maxs[3];
unsigned short firstleafface;
unsigned short numleaffaces;
unsigned short firstleafbrush;
unsigned short numleafbrushes;
} dleaf_t;
typedef struct
{
unsigned short planenum; // facing out of the leaf
short texinfo;
} dbrushside_t;
typedef struct
{
int firstside;
int numsides;
int contents;
} dbrush_t;
#define ANGLE_UP -1
#define ANGLE_DOWN -2
// the visibility lump consists of a header with a count, then
// byte offsets for the PVS and PHS of each cluster, then the raw
// compressed bit vectors
#define DVIS_PVS 0
#define DVIS_PHS 1
typedef struct
{
int numclusters;
int bitofs[8][2]; // bitofs[numclusters][2]
} dvis_t;
// each area has a list of portals that lead into other areas
// when portals are closed, other areas may not be visible or
// hearable even if the vis info says that it should be
typedef struct
{
int portalnum;
int otherarea;
} dareaportal_t;
typedef struct
{
int numareaportals;
int firstareaportal;
} darea_t;
[/code]
* fixed area bug for headnoded entities
* fixed noclip outside world view with areas
* fixed gl_lienar getting stuck
+ cull sound spatialization by area?
+ don't save level if going to a new map instead of gamemap
+ put .pak format into qfiles
- make chaingun do less damage per bullet than the machinegun?
+ check entity sound overriding in fact2
+ software glitches with areaportals
+ move spawn and free into game
+ weird blaster trails
+ make sure doors open / close areas properly reletive to sound starts
guarantee string fields are never NULL?
linksequence
client userinfo updates
software skybox rotation
unlinkentity?
make -ip work as a cvar
ip as userinfo
areaconnected game primitive
server engine manages
communication
connection establishment
collision detection
console interface
map/game transitions
cache pvs/phs expansion
falling damage
manual mipmapping on skins?
-----------------------------------------
John Carmack's .plan for Sep 25, 1997
-----------------------------------------
+ should loadgame allways unload and re-init game dll?
+ load level with spawnpoint is different than load game
+ good sound control instead of staic sounds
+ move use / drop into game code
+ texpaint autosave
+ noreadlevel cvar?
localconnect sometimes needs to be reissued
non-axial triggers
rename and g_client to p_*?
don't nextlevel from "you win"
userinfo visible to progs
too many edicts in an area for sv_world?
spawn flashes are still wrong
-----------------------------------------
John Carmack's .plan for Sep 28, 1997
-----------------------------------------
+ pain sounds?
window crunching on win95, due to order of DX operation?
-----------------------------------------
John Carmack's .plan for Sep 29, 1997
-----------------------------------------
* texpaint: backup files
* texpaint: size dialog on new skin
* texpaint: auto save
* normailized translucency level in gl
* fixed func_group entity miscount
* fixed target_changelevel use clearing
* fixed pointcontents with moving entities
* fixed spawnpoint storage
* use areas in multicast
* removed ambient sound calls
+ >512 entities
+ view kick even without move kick
+ blaster spawn point
- include sky, skyaxis as player_state variables?
+ transfer player health on each spawn and level exit
set spawntemp strings to "" instead of null?
alias save sequence number issues in qdata and texpaint
client userinfo
different blaster flash against flesh
rename edict_t to entity_t
init without IP
never same pain animation twice in a row?
map during loading bug
remove old_origin
makeuserinfo / makeserverinfo?
makeclientcommand?
auto-loop entitystate sounds?
!!! must include full path of predicted positions in fatpvs !!!
muzzle flash effect in entity_state?
jail4a iris door problem
MOTD console variable
QE4: scrub out partial detail brushes
-----------------------------------------
John Carmack's .plan for Sep 30, 1997
-----------------------------------------
* better host_speeds
* fixed bsp slowdown bug
* made Draw_Pic: bad coords not an error
* avoided double game init on loadgame
* moved serverflags into game
* fixed pause on initial connect from dumped loopback
* inventory rate bug
* client view rolling bug on level change
* RF_GLOW
* texture animation
+ more barrel explosion damage
+ archive all sensitive server cvars
+ fix inventory system
+ target_goalcounter
+ get rid of packet_entities_t as a structure
+ cinematic message name bug
+ up as jump?
+ more light!
+ weapons need to be more obvious in the world. light pulsing?
+ make first backtrack into an area with known landmarks
fade center printed text
still have color 0 problems under NT
allow game to select a client slot for connections?
combine g_player with something else
save view angles in savegame somehow
more function pointer checks in loadgame
watch out for different maxclients on loadgames
increase alias model shading level?
use userinfo_sequence
HUD strings need to be tag_game, not level
make ping time available to game?
different faces on status bar for male/female characters?
save health across levels
stuck in water currents in base2?
clamp max fly velocity
fish in ait on base3
guys shooting through force field sometimes in bunk
rename gl_mesh to gl_alias? or r_alias to r_mesh?
jump / crouch key placement
auto view centering
more edge on mipmapping in software?
-----------------------------------------
John Carmack's .plan for Oct 01, 1997
-----------------------------------------
* EF_ANIM
* got rid of precache_
* got rid of SV_Error
* !!! config strings !!!
+ are lightstyle strings being dynamically freed properly?
+ pause
+ remove SV_Error?
!!!move timedemo to server
should setmodel take an index?
smart precache of weapons?
long crawls are annoying
skin reference counting
does leak test work?
bad surface extents levels
make sound and image names include extensions?
!!! how to download implicit images ??? !!!
!!! demo recording with deltas needs to wait for full update !!!
make timeout at least a minute?
multicast_all_r for configstring should go to connected as well as active
clients
string encode SKY_AXIS and SKY_ROTATE in SKY?
it will be possible to get an index for an item not yet known
because of reliable / unreliable issues
block_until_reliable option?
supress flag on HUDs to allow cheap blinking?
rename "map" to "start"?
extra packet dumps still happening on map start
remove CL_MapEntity
move baselines into a parallel array?
don't expose svc_tent / muzzle flash numbers to game?
dropcommand cvar to restart crashed servers?
better box top walk jumping
full death cycle for player
inventory is persistant, per-client state.
no high step jump out
pain and death animations should be based on impact direction and
total damage in that frame
check on virtual alloc / commit issues
weird bmodel edge stream problem
increase numstacksurfs / numstackedges
clear sound buffer on loading plaque
-----------------------------------------
John Carmack's .plan for Oct 02, 1997
-----------------------------------------
* don't lerp blends
* sum damages for end of frame
* damage kicks scaled by health
* don't run more than one frame at a time
* fixed alias model brightness in software
* equalized light feedback value between refs
different console background
infantry melee attack?
still get stuck sometimes
mono lighting should not color alias models
put away restart game menu
rename qmenu and menu.c to something common
numeric keypad controls?
monster hearing not right?
walk into player = allways attack
are infantry fullbright flashes not working?
level to level health
lighting feedback still different in software and gl
player pain sounds
animate translucent textures
better monster sight / hearing
make sure switches are animating
echo center prints to console
PHS or PVS activated guards?
rotate with textures option in QE4 for crates
do mynoise entities leak on level transitions?
crouch strafe is still full speed
-----------------------------------------
John Carmack's .plan for Oct 03, 1997
-----------------------------------------
* game pause
* pain sounds
* save health between levels
* moved baselines to a parallel array
* software screenshot directory
* 1.4k packets only!
* map command while paused?
* pos sound overriding
* sound area testing
* seperate pvs / phs static arrays
* cleared sound buffer when disabled for loading
* PHS calculation bug
sound improvements since q1
respatializing on moving entities
sub frame start commands
looped sounds are deterministic
sounds are removed by area and PHS
looped sounds sum
muzzle flashes!
bullet impact puffa?
why can't you fire a single machinegun bullet?
avoid loading the map file twice for server and refresh
option for multicast to PVS for effects instead of PHS?
remove sbar2 sounds
centralize all communication between client and server sides
warnings for improperly looped sounds
multiple speaker entities with the same looped sound
will just increase the range
target_speaker checkbox for player locals (voiceovers)
base1 - base2 - base1 - base2 -death goes to base1
paused level to level bugs
yaw towards killer
monsters not going to ideal yaw when shooting?
blood jet muzzle flashes
iventory update on level change
keepalive messages while precaching
no red flash on deaths?
initial and final trail parts
server quit doesn't get the disconnect message out
option to have dlights backface cull
extended sound bytes
release mouse when paused?
nver let server be connected without local client
!!! possound needs to also take an entity number !!!
slow water wading
-----------------------------------------
John Carmack's .plan for Oct 04, 1997
-----------------------------------------
* map_noareas
* target_speaker
handle bmodel origins on client side, search for good area
check localsound pos starts
make the server read the demo configstrings and baselines
and spit back to the client like normal
water wading sounds
sound streaming option
remove cl_mapentity
sync camera pain motion to sound length?
give all sounds a max volume area isntead of instant diminish
quakeworld style shotgun handling
-----------------------------------------
John Carmack's .plan for Oct 05, 1997
-----------------------------------------
Developing for windows is not fun. We are having a lot of trouble getting good solid compatability across all the systems we are testing on.
When it works right, it just pops right into full screen mode with sound and network just like the dos version, but we are still chasing problems on several systems. Sigh.
* !!! autolooped entity sounds !!!
make all tools into 5.0 projects
combine SZ_ and MSG_
allways mkdir gamedir?
pause dumps packets?
clear all background all the time flag
player physics
MD4 each map file?
print version number on console bottom
select a different cd track if all goals accomplished
get rid of alphalight
-----------------------------------------
John Carmack's .plan for Oct 06, 1997
-----------------------------------------
* larger bsp token length
goal sound, secret sound, help sound
remove pushmatrix/popmatrix
less shademodels
sound mixaheads
flies should be a sound field
trinity: two pass texture checking to avoid thrashing?
lose links and $ macro expansio?
lose Com_SetServerState
blinking flags on huds (blink F1)
!!! rotating object view changes not in yaw !!!
save configstrings in level to get lightstyles
better armor feedback
armor sounnd?
-----------------------------------------
John Carmack's .plan for Oct 07, 1997
-----------------------------------------
* pitch snap clamping
* clamp at 89
* kill sounds when loading plaque
* no fov or null pic problem during prep refresh
* wounded faces
die then bring down console over menu bug
death
windows key
make sprite files text format?
all explodables and breakables should be pre-broken in dm
flash stats on change?
send email to j sturges
inventory names
super tank skin
variable delay on centerprint
timedemo
demo tests
flies as entity sound
release mouse when paused?
centerview
peak to peak view bobbing
counter items
infantry skins
menu sounds
secret sound
goal sound
sound when low on health?
respawn muzzle flash event still wrong
falling damage
rotating sky in software
color 0 on NT
transparent water insides
-----------------------------------------
John Carmack's .plan for Oct 08, 1997
-----------------------------------------
* fixed entity numbers 512-1024
* combined baselines and oldorgs
* demos working again
* MAX_SFX bug
spawn invisible allways starts at 256, so visible get bytes?
check goal counting
any key puts away help?
muzzle flashes
make cl_entities dynamic?
removed mergedemo on client
remove all client demo playback
finish savegame / loadgame UI
finish cinematic sound
check demo fopen spawning for cddir
weird palette issues?
more red
minimum health
infantry muzzle sounds?
remove blaster hit flashes on flesh
no savegame when dead
mine2 dissapearing problems
use key problems
qrad: infinite styles on face
allways have two secrets
allways select new items
do demos need a precache command?
timegraph
!!!save lightstyles in savegame!!!
!!!save areaportal state in savegame!!!
are loadgames doing 10 second prerun?
move say and say_team into game
sentity_t gentity_t
more blood
areaportals in software - bad sort keys?
-----------------------------------------
John Carmack's .plan for Oct 09, 1997
-----------------------------------------
* removed MAX_PACKET_ENTITIES limit
* used areas for beam culling
* centerprint to non client not error
* don't rotate roll when pushed by entities
* areaportal fragments in software
* F_CLIENT fix
* KEY_ANY fix
* save areaportals
* save lightstyles
* fixed secret double counting
* up as jump
color 0 on NT
water wading speed
water jump out
centerview
no savegame when dead
mroe damage blend
putclient in server shouldn't reference weaponmodel
userinfo issues
IP cvar for servers
IP userinfo for clients
remove sv.viewpos?
make max_entitites a noset cvar
don't use PHS?
up / down issues
broadcast centerprint
intermissions
flickery lights
free mouse when paused
-----------------------------------------
John Carmack's .plan for Oct 17, 1997
-----------------------------------------
Somehow we managed to convince the mesquite city services and police department to let us take our cars down to the municipal airport and run them down the runway to get radar speed numbers. Is that cool, or what?
So, how fast can we go on a 6000 ft runway and still stop before running off the end?
John Cash's M3 just barely hit the 135 mph speed governer.
Bear's turbo supro hit 144
My F40 hit 165
My TR's left turbo exploded at 160 mph :(
Adrian, Todd and Paul couldn't make it, so we didn't get viper, vette or porsche numbers.
It took less than 2000 ft for the TR to do 160. We were fully expecting to do 200 mph in 4000 ft if things had held together.
We have a bunch of video and sound footage that we are going to digitize eventually. We made one run with a police mustang chasing after my F40. Guess who won.
The F40 is a very, very durable car. I made six runs around 160 mph, and it didn't even fade. Same thing on a racetrack. Lap after lap without any changes. My TR makes 1100 hp for twenty seconds, then explodes..
-----------------------------------------
John Carmack's .plan for Oct 19, 1997
-----------------------------------------
I hope everyone is enjoying the quake 2 test.
Its always hard to release a version of a product that you know isn't in its final form. There are plenty of things that are getting better every single day, but we need to chop it off at some point to let everyone test it out.
We will do another demo after we finish the full retail product, so if you don't like looking at preproduction stuff, wait for that one.
Still, I am pretty happy with the test. I think Quake 2 is definately the most cohesive game we have ever done.
Don't worry - just because the test doesn't have multiplayer in it, it doesn't mean that we haven't been thinking about it. Many features in the quake 2 architecture are going to enable a whole new level of net play. It will take a few months after the full release for all the potential to start showing through, but just you wait!
The biggest changes to Quake 2 are internal. Anyone doing modification work on Quake is going to be ecstatic when they get to work with quake2. The game dll source code and all the utilities (including the OpenGl map editor) will be released shortly after the game hits store shelves.
-----------------------------------------
John Carmack's .plan for Oct 20, 1997
-----------------------------------------
Many of the comments about the Quake 2 test are already being addressed. We expected quite a few of them, but the test has served its purpose of bringing in some good feedback that we couldn't have predicted.
The final game will definately be better as a result of the test.
However, it certainly won't please everyone. I am confident that the majority will think that Quake 2 is significantly better than anything we have ever done before, but even if we please 80% of our potential customers, that will still leave a couple hundred thousand people thinking that we let them down.
I suppose that I have it the easiest there - I can always defend my technical decisions with specific discussions of my evaluations of the tradeofs that led me to the paths I chose. In fact, in a large number of cases when someone suggests something, I can actually say "Tried it. Didn't work as well."
Defending level design, artwork, or sounds is a lot harder. We can't even always agree here at id on many of these issues, so we know for sure that we can't please all the users simultaniously. All we can do is put talented people on the job and have confidence in their abilities.
Note: Q2TEST DOES NOT INCLUDE ANY HIGH QUALITY SOUNDS! That would have added another 15 megs to the demo size. Selecting high quality sounds just upsamples the existing 11khz / 8 bit sounds. There is a significant quality increase (at a slight speed and memory cost) with the full production sounds.
Quake 2's goal is to be the best first person shooter ever. We are trying to evolve a genre, not move to a different one. If you don't want a game that mostly consists of running around and killing things, you will be dissapointed. We are trying to be cohesive, but not deep. I have high hopes for the games that are atempting to aply our technology to other genres, but don't look for it in Quake 2.
A quick plug:
if you have any interest in programming, you should look at Michael Abrash's Graphics Programming Black Book Special Edition. It has just about everything he has written, from the ancient work on optimizing for the 8086 (still interesting to read) to the articles written during quake's development.
I personally learned a lot from Michael's early articles, and I was proud to contribute to the later ones.
http://www.amazon.com/exec/obidos/ISBN=1576101746/3226-6070174-877180
-----------------------------------------
John Carmack's .plan for Nov 01, 1997
-----------------------------------------
* interpolate prediction error
* fixed farthest respawn
* removed backspeed
* no pickup weapons when dead!
* multiple crosshair pics
* fixed dropping items in wall
* disabled auto weapon switching in deathmatch
* respawn_time
* mroe precaches
* removed doubles
+ pitch around bug
+ ping calculation
+ are demos broken with prediction?
+ no footsteps if moving slow?
+ kill self command
+ no toss weapons in wall
+ footstep doubletap
+ clear gib flag on respawn
+ faster weapon deop times
precache talk wav
qbsp: MAX_MAP_AREAS when leaking?
flies effect on hyperblaster???
no slide under staircase
make blaster bolt move faster
BECOME_FLASH effect?
make dedicated server sleep
connect to other server while playing bug
bit code net messages?
win95 joining
changeweapon work better when out of ammo
pop in maps?
highlighted numbers
sort image_t lists?
switching rules
brain effect
monsters source shots before testing line of sight
secret doors
obituaries from monsters
warp gate effect
teleporter effect
increase max switched lights
max lightstyles bug
shorter wav latency
beep beep on pc icon
blood jets from pain skins
gibs shoot up out of lava
savegame in water bug
font outlines not sharp in gl
weapons vanishing on toss?
longer pause after death before respawn
temp invulnerability after respawn?
telefrag not always working?
step up in water?
clear powerups on death
don't hold a grenade when none left
l_health item precache?
-----------------------------------------
John Carmack's .plan for Nov 02, 1997
-----------------------------------------
* fixed pitch clamping
* Com_PageInMemory
* fixed menu cursor time
* net connect when playing bug
* custom skins
* fixed server update without game update bug
+ bump version numbers
+ don't go to half console until connected
+ delay before firing bfg
+ stairup allows wall climbing now
+ previous frame issues
+ don't copy all of frame.packetentities
no server pause in dm
normalize skin texture coords for software
splashing sound when swimming in water
ipx
software underwater surfaces
nopredict option at server
scroll inventory
smooth step up
remove rand1k
fire func_explosive targets when starting in deathmatch
-----------------------------------------
John Carmack's .plan for Nov 03, 1997
-----------------------------------------
* fixed divide by zero in kickback
* fixed overflow
* fixed walkmap up slopes
* bumped versions
* grabbed all cinematics
* fixed abort intro issues
* kill command
* fixed server status command
* more weapon precaches
* noexit by default
* autoremove some stuff in deathmatch
* make game initialization like - sound -
* pumped message loop during caching
* client persistant data
+ no drop weapons without ammo
+ no footsteps when walking
+ saved across kills
+ saved across levels in single player
+ client levelstate
+ clear client times on level change
+ shouldn't be dumping unreliable messages
+ palette changes on cinematic
laod game should throw loading plaque immediately
not pausing when menu is up!
"don't need" sound for no pickup?
min_intermission cvar?
run key should be a toggle
dropped items respawn
auto use items
powerup sounds
print sound precache pacifiers
sendkeyevents during loading?
better pingservers
alias models are lit outside ofdlight ball
sv_shownet
"killed by" icon on scores?
don't allow dedicated without deathmatch
map transitions, but gamemap doesn't?
intermission spots with deathmatch
blinking health indicator
blinking computer indicator
move swapbuffers and add flush to glquake?
chaingun sounds off on NT?
test ping on serial port to serial port connection on win95
console prog commands
lower the scoreboard readout so you can read the obituary bessage
-----------------------------------------
John Carmack's .plan for Nov 04, 1997
-----------------------------------------
* allowed bad sky textures
* cinematic tweaks in gl
* high res skins
* fixed duck speed issues
+ names are messed up
+ not disconnecting cleanly?
+ cinematic GL wrapping problem
+ cinematic quality issues
+ cinematic sound
+ cinematic end frame marker
+ switch sound to high quality for cinematic
+ is idlog aborting early on menu?
+ dropping items makes them respawn in deathmatch
+ crouch strafe is faster than forward
+ sink in plats bug
+ lower paused icon on screen
+ min firing ammo for dropping weapons
+ spawn explosions with a random yaw
gun puff animations wrong?
no status bar during intermission
sv_showcmds
are sky images freed properly?
wading sound
use 16 to 8 table is ref_soft for tga loading?
savegame off pak file
demo file parsing from pak?
check replace alpha value for mcd hack
screen update timer for software opengl?
instant items
item sounds
include texture source size in texinfo so other scaled versions can be made?
are cinematics using color 0?
send pak checksum to server?
fix dedicated_start
print dm rules on connect?
blink f1 and play sound
skill values!
loadgame from console
input based demos for profiling
-----------------------------------------
John Carmack's .plan for Nov 05, 1997
-----------------------------------------
* fixed sink into plat bug
* fixed scoreboard display between deathmatch levels
* seperated game dll definitions
+ clamp low cin times
clear angles on loadgame
check client entering during intermission
seperate headers for monsters and players
-----------------------------------------
John Carmack's .plan for Nov 06, 1997
-----------------------------------------
* s_testsound 1
* fixed streaming sound on 95
* streaming sound at full volume
* removed multiply from mixing
* khz change for cinematics
* blaster precaches
* fixed cinematic from pak streaming
* don't use primary sound buffer option
+ precache blaster
+ dropped grenades on death shouldn't respawn there..
set hostname by ip hostname
timegraph not right
rename map to start
check entire game without asm code
no mouse cursor when fullscreen
die with grenades needs to stop ticking sound
high quality sound directories
-----------------------------------------
John Carmack's .plan for Nov 07, 1997
-----------------------------------------
* flag reorg
* teleporters
* put holdangles into pmove.pm_type
+ pm.touchents holds duplicates
+ damage anything flag
+ precache chat sound
+ teleporters at player spawn points
+ remove rocket fragments in dm
rename entity_t to rentity_t ?
teleport sequence bit to make ef_teleport reliable?
turn any event into a temp entity? (with or without angles)
unify sound starting as temp entities?
is time being over quantized by timegettime?
order events by priority
login / logout as events?
all sound channels as extra events?
trinity: objects should have enabler inputs as well as multiple impulse targets
-----------------------------------------
John Carmack's .plan for Nov 08, 1997
-----------------------------------------
* make random respawn option default and work
* don't drop empty weapons
* teleport angles
* teleporters at player spawn points
* fixed telefrag self on respawn
* fixed userinfo on initial entering
* precache land sounds
* don't change console height until connection packet
* a disconnecting client shouldn't generate a badread
* remove rocket fragments in dm
* damage anything flag
* don't call duplicated pm.touchents
* client parse entities array
* no weapon toss in single player
sound dies after several hours?
scoreboard faces
show killed by face on scoreboard
deal with old_origin properly
delay cdtrack play until connected
can't escape out of loadgame menu when dead?
-----------------------------------------
John Carmack's .plan for Nov 09, 1997
-----------------------------------------
* no client pmove at all with prediction off
* railgun crashes
* fixed rub stuck bug
+ gravity in pmove
+ control config is messed up
+ seperate client event processing from parsing
+ are baselines not working right?
make-item-selected command
notched look up / down commands
keyboard look
map name is messed up on start server
deathmatch character weapons
unify uses of ent / client / etc in source?
no player actions when paused
check incoming_acknowledged for pings
master servers
dm respawn isn't clearing pitch?
teamed teleport destinations
teleport angles
# form for kicking players
make showinventory a stat bit, like layout
unify all data file references for easy downloading?
how to handle different quality versions?
if crushed in dm, respawned outszide world?
shadows
rename entity_t->flags to renderfx
checksum client to server messages
smooth out step up
does spectator work?
trinity: coarse radiosity source lattice for dynamic lights?
oldorigin issues
replace with a previous state send?
-----------------------------------------
John Carmack's .plan for Nov 10, 1997
-----------------------------------------
* qe4 bad class parse bug
* grenade bounce generates two sounds
* playerstate delta
* qdata variable sound rates
check the replace mode alpha bug
replace all muzzle flashes with events
lower railgun
-----------------------------------------
John Carmack's .plan for Nov 11, 1997
-----------------------------------------
* cinematic playback at variable sound rates
* qdata multiple video in single file fix
* only one token huffman bug
* demowaiting
* allowed individual reliable overflows
* make all clients invisible at intermission point
* mask high bits in client_t->name
* full delta compression
don't show paused plaque in dm
check all serverinfo flags (skill, nomonsters)
crunch the scoreboard data a lot
dedicated net thread
random-not-nearest option
"public" variable
check the "using previous_origin" notes
echo chats to console?
weapon icon when hand is centered
game skills
better console keyboard editing
are demos broken with current delta compression?
should client string commands be ties to usercmd_t, to fix drops?
why is ref_gl.dll as large as it is?
allow round up and >256 textures on gl
mouse during cin
echo chats to console
is the dedicated server sleeping?
get mins/maxs from pmove
warn at startup if any spawn point is in a wall
make teleport pads glow?
never make water solid for entity cull purposes?
get sound position needs to check the valid count on entities
make secondary sound buffers the default?
pak file sorting by traces?
no colored lighting with mono lightmaps!!!
option to make picking up items not select
change stretch-raw to a dedicated full screen blit for better performance?
bsp to do:
=========
robustness
water problems
increase MAX_MAP_ENTSTRNG
allow any number of light styles
MAX_MAP_AREAS when leaking?
-----------------------------------------
John Carmack's .plan for Dec 01, 1997
-----------------------------------------
Quake 2 has mastered.
Where we go from here:
Point release.
We should have a Quake 2 point release out shortly after the game gets in your hands. We intend to fix any bugs that turn up, improve the speed somewhat, and optimize for internet play in various ways. We will also be making several deathmatch only maps.
Deathmatch in Q2 has gotten a lot of lan testing (Thresh, Redwood, and Vik Long helped quite a bit the last week with tuning), but not much internet testing. There are probably gaping holes in it, but we will address them soon.
The deathmatch code in the shipping Q2 is also not designed to hold up against malicious users - there is no protection against clients being obnoxious and constantly changing skins, chat flooding, client-side cheating, or whatever.
Q2 does checksum maps on the client side right now, so cheater maps won't work like they do in Q1, but cheater models and skins are still possible. I have some plans to combat that in the point release, but there are a lot of forms of cheating that can be implemented in proxies that are fundamentally not detectable. I can make it very painfully difficult for people to implement such things, but a very clever person with a dissasembler just can't be stopped completely.
The server code and network protocol should be able to support ultra-large player counts, but I know I need to do some low-level work to get around operating system buffer limits before it will actually work. We will test at least a hundred players in a giant map for the point release, but we won't actually address the issues of making a rational game at that level (chat hierarchies, team spawning, etc). I am very much looking forward to seeing what the user community creates on that foundation.
It is likely that the point release may have incompatable network protocols and savegames. Fair warning.
Q2 Demo.
After the point release, we will be making a new demo release. If you experienced compatability problems with q2test, or were unsatisfied with the quality in some way, you should look at the demo. The final product is much improved.
Q2 Ports.
We are commited to Win32 Alpha, Linux, irix, and rhapsody in that order. It is likely that a bunch of other ports will come later, but no promises. The presence of hardware-accelerated OpenGL on a platform will improve it's odds a lot. Zoid will probably prioritize Q2 CTF over other ports, so hold off on bugging him about ports for a while.
Development tool release.
I will basically be making publicly available a subset of the directory tree that we will deliver to our licensees. All the utility source code, the game dll source code, and probably some example source media - .map files, artwork, model source, etc.
Q2 mission pack.
Most of the company will be working on a mission pack while Brian and I write tools and technology for trinity.
Trinity.
I am going to rapidly wean myself off of working with quake so I can concentrate fully on new directions. The evolution of the Q2 codebase will be left to John Cash (until the mission pack ships) and Zoid.
Everyone should keep in mind that any next-generation game that we produce is a LONG way off, so don't start getting all worked up over it, ok?
For the curious, it does look like java is going to start playing a significant role in our future projects. All of the lightweight utilities will be java applications (some requiring OpenGL bindings). The heavy duty number crunching utilities will probably stay in C. It is still unclear how much of the game framework and the level editor we can get away with doing in java.
-----------------------------------------
John Carmack's .plan for Dec 02, 1997
-----------------------------------------
A couple things I forgot to mention:
DOOM source. Still planned to be released Real Soon Now, but there is some work that needs to be done on it first to remove the sound engine, which was written by someone else.
Our Quake editor. It will be released with the tools, but it really isn't going to be all that usefull to many people. Most people will be better off with one of the actively supported editors designed for normal machines.
There is no documentation (Steve Tietze at Rogue has talked about writing something, though). It is designed to run at 1280*1024 resolution on a fast, fully-compliant OpenGL driver. It was designed for high-end boards like intergraph realizm, 3DPro, and Glint boards, but it also runs ok on 8 mb consumer boards like the permedia II and rendition V2200. It will NOT work with voodoo or powerVR. It is unlikely to work with voodoo rush, because of framebuffer size limits, but it might work at a low screen resolution. It might be workable on RIVA cards if they do some fancy work disposing buffers between window renderings (they are a 4mb card, but the textures can stay in AGP memory, so it will almost be enough). I'll work with them if they want to give it a try.
Right now, only 3Dlabs has a full opengl driver on win-95 (and it is a little flaky). All the other cards would require you to run NT. Over the next several months, most of the major vendors should be releasing full OpenGL drivers that work in '95, but there are no firm release dates.
A comment to the people complaining about the release not having Their-Favorite-Feature:
A software project is never, ever completely finished. If you wait until EVERYTHING is done, you won't ship at all.
Would it have been the right thing to delay releasing Quake 1 until I had written the glquake code and the QuakeWorld code? Or we had gotten Paul to build us all new models? Or we had made all new maps that hang together thematically?
If we had, we would be releasing Quake right about now. It would be a much better game (it would be Quake 2), but all of the enjoyment that everyone has gotten from Quake would have been lost. It would have been the wrong decision.
Quake 2 is great, and it will get better yet after its release.
A reminder about "John Carmacks":
Anyone claiming to be me on IRC is lying. I have never been on IRC, and if I ever choose to, I will mention it here first.
If you get an unsolicited email from "John Carmack", the odds are high that it was spoofed. Every couple days, I get a mail bounce from someone who messed up on a spoofed mail, and I often get confused responses from people that I have never mailed.
-----------------------------------------
John Carmack's .plan for Dec 09, 1997
-----------------------------------------
BIG BUG IN Q2 NETWORKING!
If you run multiplayer servers, download:
ftp://ftp.idsoftware.com/idstuff/quake2/q2_306.zip
A serious bug got through.. I thought the QuakeWorld master server code was completely disabled, because I was planning on putting a modified architecture in place in the point release. It turns out that the code is still in there, sending heartbeats to a unix machine here at id that isn't even running a master server.
That wouldn't normally be an issue - a packet every five minutes from all the servers.
Except...
Cyrix has a new processor that is significantly faster at single precision floating point calculations if you don't do any double precision calculations anywhere.
Quake had always kept its timebase as a double precision seconds value, but I agreed to change it over to an integer millisecond timer to allow the global setting of single precision mode.
We went through and changed all the uses of it that we found, but the routine that sends heartbeats to the master servers was missed.
So, instead of sending a packet every 300 seconds, it is sending one every 300 MILLISECONDS.
Oops.
To a server, it won't really make a difference. A tiny extra packet three times a second is a fraction of the bandwidth of a player.
However, if there are thousands of network games in progress, that is a LOT of packets flooding idsoftware.com.
So, please download the new executable if you are going to run any servers (even servers started through the menus).
ftp://ftp.idsoftware.com/idstuff/quake2/q2_306.zip
This isn't the real point release - there are no new features or bugfixes. I just went back to the release codebase and recompiled with one function commented out so we wouldn't have to worry about introducing new bugs with our current untested code.
Btw, all bug reports should go to Christian (xian@idsoftware.com), NOT to me, Brian, or Cash! We need a central point to funnel things through. Hopefully we can set up a web page or something to make public what we know about, so we can cut down on email traffic.
-----------------------------------------
John Carmack's .plan for Dec 11, 1997
-----------------------------------------
The Quake 2 public code release is up at:
ftp://ftp.idsoftware.com/idstuff/quake2/source/q2source_12_11.zip
This source code distribution is only for hard-core people that are going to spend a lot of time pouring over it. This is NOT a how-to-make-levels-for-q2 type dsitribution!
This should keep a bunch of you busy for a while. :)
-----------------------------------------
John Carmack's .plan for Dec 23, 1997
-----------------------------------------
The DOOM source is up.
Merry christmas!
ftp://ftp.idsoftware.com/idstuff/source/doomsrc.zip
---- contents of README.TXT -----
Here it is, at long last. The DOOM source code is released for your non-profit use. You still need real DOOM data to work with this code. If you don't actually own a real copy of one of the DOOMs, you should still be able to find them at software stores.
Many thanks to Bernd Kreimeier for taking the time to clean up the project and make sure that it actually works. Projects tends to rot if you leave it alone for a few years, and it takes effort for someone to deal with it again.
The bad news: this code only compiles and runs on linux. We couldn't release the dos code because of a copyrighted sound library we used (wow, was that a mistake - I write my own sound code now), and I honestly don't even know what happened to the port that microsoft did to windows.
Still, the code is quite portable, and it should be straightforward to bring it up on just about any platform.
I wrote this code a long, long time ago, and there are plenty of things that seem downright silly in retrospect (using polar coordinates for clipping comes to mind), but overall it should still be a usefull base to experiment and build on.
The basic rendering concept - horizontal and vertical lines of constant Z with fixed light shading per band was dead-on, but the implementation could be improved dramatically from the original code if it were revisited. The way the rendering proceded from walls to floors to sprites could be collapsed into a single front-to-back walk of the bsp tree to collect information, then draw all the contents of a subsector on the way back up the tree. It requires treating floors and ceilings as polygons, rather than just the gaps between walls, and it requires clipping sprite billboards into subsector fragments, but it would be The Right Thing.
The movement and line of sight checking against the lines is one of the bigger misses that I look back on. It is messy code that had some failure cases, and there was a vastly simpler (and faster) solution sitting in front of my face. I used the BSP tree for rendering things, but I didn't realize at the time that it could also be used for environment testing. Replacing the line of sight test with a bsp line clip would be pretty easy. Sweeping volumes for movement gets a bit tougher, and touches on many of the challenges faced in quake / quake2 with edge bevels on polyhedrons.
Some project ideas:
Port it to your favorite operating system.
Add some rendering features - transparency, look up / down, slopes, etc.
Add some game features - weapons, jumping, ducking, flying, etc.
Create a packet server based internet game.
Create a client / server based internet game.
Do a 3D accelerated version. On modern hardware (fast pentium + 3DFX) you probably wouldn't even need to be clever - you could just draw the entire level and get reasonable speed. With a touch of effort, it should easily lock at 60 fps (well, there are some issues with DOOM's 35 hz timebase..). The biggest issues would probably be the non-power of two texture sizes and the walls composed of multiple textures.
I don't have a real good guess at how many people are going to be playing with this, but if significant projects are undertaken, it would be cool to see a level of community cooperation. I know that most early projects are going to be rough hacks done in isolation, but I would be very pleased to see a coordinated 'net release of an improved, backwards compatable version of DOOM on multiple platforms next year.
Have fun.
John Carmack 12-23-97
-----------------------------------------
John Carmack's .plan for Dec 25, 1997
-----------------------------------------
We are going to release a new quake 2 executable that fixes the malicious server crashing problems Real Soon Now. It also fixes a ton of other problems that have been reported, so we are going to have to give it some good testing before releasing it.
John Cash has two kids that would lynch him if he came in and worked on christmas, so we certainly won't be able to get a release candidate together before the weekend. I am fairly confidant we will have it released to the public on sunday.
I have been spending most of my time on trinity research but I have still made quite a few fixes to Q2. John Cash has made many more (he is just finishing up the IPX coding, among other things).
I have been doing a lot of testing over a proxy that gives me a very bad ping (400 - 800), so I was able to find and fix two significant errors with the prediction code.
The reason why you get a jerk when running forward and firing rockets, blasters, or grenades is that the client side prediction code was blocking you on your own missiles.
The jerky behavior on plats was due to a subtle error in the prediction error interpolation. A prediction error was causing oscillations as long as your latency, instead of smoothing out over just 100 ms. The plats are now smooth as long as you aren't dropping packets, and other mispredictions are also handled much better.
There are still a lot of other things that will be fixed in an upcoming release, but this will definately be an executable worth grabbing.
My fixes:
* zombies aren't being removed properly
* joystick not in menu
* classname for rockets and bolts
* no screaming when invulnerable and in lava
* lowered water blend values
* clear powerups when dead (no more breather sounds)
* only play "computer updated" three times max
* mapname serverinfo now updated properly
* changed "rejected a connection" to "Server is full"
* made console "rejected a connection" a developer only message
* made WSAWOULDBLOCK warning silent
* max 10 packets/second during connection process
* set cl_maxfps to 90
* increased loading plaque timeout value to 120 seconds
* paused not default to 1
* no savegame in deathmatch
* fixed ; binding from menu
* no crouch when airborne
* removed half-baked $ macro expansion
* pause on landing before re-jump (fixes no fall damage bug)
* public server framework
* no ; comment in config files
* teleporter events
* lower hyperblaster damage
* don't use PORT_ANY for clients!
* fix the entity number thing here
* don't re-check CD after the first time
* auto cddir from cd scan
* dissallow kill from intermissions
* faster rockets
* less bfg effect damage
* remove packet command from client
* strip trailing spaces on cmd_args
* added protocol to serverinfo
* used CMD_BACKUP instead of UPDATE_BACKUP for phone jack
* don't predict clip into your own missiles
* good netgraph
* validate userinfo for semicolons and quotes
* don't copy savegames on dedicated servers
* also check current directory for game dll loading
* changed connect packet on client to differ from server
* bump protocol version
* fixed error interpolation on plats
* only respawn with attack or jump
* fov as a userinfo
* show weapon icon if fov > 90
-----------------------------------------
John Carmack's .plan for Dec 27, 1997
-----------------------------------------
The 1.07 patch is out:
ftp://ftp.idsoftware.com/idstuff/quake2/patch_07.zip
Please mirror and distribute this.
When submitting bugs, make sure you say that you already have the 3.07 patch.
Christian will go through and update the bug page when he gets back from vacation next week.
This release does not fix all known problems. We intend to have another release in a few weeks.
-----------------------------------------
John Carmack's .plan for Dec 28, 1997
-----------------------------------------
** Dec 28, 2:55 AM CST **
There were a few problems with the 1.07 patch:
Bodies stuck under doors caused a repeated explosion effect. Timedemo was broken. The servers crash about once an hour under full load.
I have the first two fixed, and I hope the third. The four servers at Id are running a new executable.
If the servers don't crash in the next hour or two, I'll put another release out.
** Dec 28, 5:00 AM CST **
Ok, two hours without a crash on four servers.
Here is a new patch:
ftp://ftp.idsoftware.com/idstuff/quake2/patch_08.zip
3.07 and 3.08 can interoperate fine. All servers should upgrade to 3.08, but if you gravved the 3.07 earlier today and only play as a client and don't need timedemo, you don't nned to upgrade.
** Dec 28, 5:00 PM CST **
No crashes on any of the servers!
A few comments on some reported problems:
You have to press the "attack" button to respawn in deathmatch now. This allows you to chat and go into the menu. I have got several mails from people that are typing "kill" or reconnecting to servers after they die...
Old savegames will NOT work with the patch. Just cheat yourself to aproximately the same place you were before. The game included config files for starting off at each unit. You can exec one of those to get you close, then do "give" commands if you want to be more precise. (bigguun.cfg, boss.cfg city.cfg, command.cfg, factory.cfg, hangar.cfg, jail.cfg, mine.cfg, power.cfg, space.cfg, warehouse.cfg).
I think several people are failing to get the gamex86.dll into the baseq2 directory. if "fov 120" doesn't change your field of view, the server doesn't have the right gamex86.dll.
** Dec 28, 9:30 PM CST **
If Quake2 is crashing on you after upgrading, it is probably because you still have the gamex86.dll from q2test in your quake2 directory. The latest quake2.exe just started looking in the exe directory as well as the game directory to make debugging easier, and it brought out this problem. You should only have gamex86.dll in baseq2 unless you are doing specific development.
I had a version check in there, but I never bumped the game api version, so it was ineffective.
We are going to release yet another new version tonight.
The big news is that the modem connection and level changing problems are fixed. They should have been fixed in 3.07, but a timing error kept it from functioning.
I also found the "no such frame" warnings that scrolled by under some circumstances. BFG gibbing crouching people would cause it.
There are several other fixes in the menu and renderers as well, so everyone should upgrade.
We are testing with 3.09 on our servers now, but I want to make an incompatable change before releasing:
Right now, any client can send a "connect" message to the server and grab a client slot. If they are the wrong version, they will tie that slot up until they time out ot abort the connection process.
I am going to force clients to send their version number with the connection request, so that bad clients will never take up slots.
That will require everyone to upgrade to 3.09 to play.
I apologize for the flurry of versions, but this was a forced set of releases due to the server attacks, and lots of people are on vacation here. It certainly could have been tested better, but I thought it better to try and get something out ASAP.
Check back in the morning for a new version...
BTW, we will release the new gamex86 source code after we are convinced that we aren't going to be making another patch for a couple weeks.
-----------------------------------------
John Carmack's .plan for Dec 29, 1997
-----------------------------------------
** Dec 29, 2:25 AM CST **
new version:
ftp://ftp.idsoftware.com/idstuff/quake2/q2-309.exe
This one has an install that makes sure things get where they need to...
** Dec 29: 2:30 PM CST **
The only widely reported problem with 3.09 is that the video playback is choppy. The fix for the modem connections reduced video playback to 10 fps. Its a one line fix, but I'll hold off on another version until a few more things accumulate.
I am curious what the breakdown of opinion is on the rapid patch releases. If one of the polling websites would pose the question, I would apreciate it.
A more liesurely patch release would allow us more testing time, and some problems (like this cinematic bug) would certainly be killed before the public saw it, but I definately found a couple things from the public that no amount of testing on our machines would have found. Some things only showed up with 48 people playing on our servers for several hours.
Once again, we really didn't have a choice this time because of the server crashers, but we are planning another release in two to three weeks.
I am happy to produce new versions fairly rapidly, rather than at monthly intervals, but I know that many people are getting a little irate at having to download new patches. There is a simple solution -- if you don't want to be on the bleeding edge, wait a week after a patch is announced and see how it is working for other people.
What finally helped me get to the bottom of some things was just getting people with problems we couldn't reproduce to call me and let me send them executables by email until I figured out what was going on. From now on, if you send a detailed problem to me, include a phone number and times when you can be reached. I'm not tech support, so you certainly can't count on a response, but if you have a nice repeatable case of a problem that is high priority for us that we can't reproduce otherwise, your personal help may be usefull.
BTW, does anyone know why Quake 2 became a hacker target? I can keep fighting attacks, but spending my time there doesn't help anyones game, and there are a bunch of things that fundamentally can't be stopped if people really set their mind to messing up the servers.
** Dec 29: 8:00 PM CST **
Please cool it a bit with the email to me unless it is really important. I'll never get trinity done with the email pouring in the way it is right now...
We have rebuilt the 3.09 patch with a new version of the install program. Some people were not able to run the installer because a temp directory wasn't setup correctly. There are NO OTHER CHANGES in this, so if you were able to install the last 3.09, don't bother getting this one.
ftp://ftp.idsoftware.com/idstuff/quake2/q2-309-2.exe
-----------------------------------------
John Carmack's .plan for Dec 30, 1997
-----------------------------------------
Until we release the new gamex86 source code, if you want to make mods work with 3.09, change GAME_API_VERSION to:
#define GAME_API_VERSION 2
and recompile the mod.
This will let it run with the 3.09 servers. The API didn't actually change, I just had to bump that version number so that we could detect the old q2test dlls still hanging around.
-----------------------------------------
John Carmack's .plan for Dec 31, 1997
-----------------------------------------
** Dec 31: 4:40 AM: **
A user just reported having their net quake problems go away when they killed ICQ. I suppose it has never been stated directly, so here goes:
Quake needs all the bandwidth that a modem connection provides to play well. Any other program accessing the internet is going to cause a degredation in gameplay, sometimes severe.
So quit IRC, ICQ, email, and web browsers before setting out for serious net play unless you have ISDN or better.
** Dec 31: 12:41 PM CST: **
I just spent a few hours working with a quake player that still couldn't net quake with 3.09.
It took a while, but I finally understand what is going on.
He could play net games on his local lan, but when he tried to connect to remote servers, it would always fail and timeout midway through the connection process, or at most a few seconds into the game.
The situation was that there was a small network of computers connected to an ISDN router that did address translation.
Address translation allows multiple computers to use the internet through a single TCP/IP address. This is accomplished by having the router perform some "invisible" port and ip renaming on everything that goes out.
I think that is a rather evil thing for a router to do, but I suppose I can see the incentive from an address pressure viewpoint.
Routers know when TCP streams begin and end, so they make sure the port mappings stay constant through the entire thing, but quake uses UDP packets (anyone who suggests using TCP for a realtime game does not understand how the error recovery works), and the router apears to be making the incorrect assumption that UDP is only used for simple request / response protocols.
The router changes the UDP port while you are playing.
Grrrr.
Now, a smarter router would only change the port numbers when it was actually forced to by a collision, which would only be when a connection was first opened, and everything would work out ok.
After I understood what was happening, I could devise a fix for it. My simple fix was to make the server simply ignore the port number for client comparisons, and assume that if a packet came from the same IP address, then it is the same player even if the port number changed. That worked, and he was able to connect in to my modified server.
That has the distinct drawback of making translating routers or proxies that do the port mapping correctly unusable by more than one player at a time.
I could fix it completely by including a sort of port number in each message, and having the servers match and update UDP ports based on that. That would work fine, but at the cost of adding a byte or two to everyone's packets to help out people with bad routers. You wouldn't be able to tell a difference, but its the principle of it...
I could make a server side cvar to force port fixing on, but that would still not work for one class of users or the other.
I could make it client settable and have the client tell the server on connect that it needs port fixing. That would work with no bandwidth cost to anyone, but it would require users to know that if they can't connect to servers, then they should try to use the fix translation option. Unfortunately, I bet that there are some routers that exhibit this problem much less often. A drop every ten minutes would be hard to attribute.
I could make port fixing on by default, but if anyone is on a translated lan and another person tries to start a net quake game to the same server then they will both collide and crash and burn.
I am probably going to add the extra bytes to every packet. Being automatically robust on more people's systems is probably worth a microscopic loss of bandwidth. Two bytes is under one millisecond of ping on a modem.
If there is some magic range of port values that I can use to make these routers act better, let me know.
These changes will break the connection protocol again, so I am going to hold off on the patch for a while.
-----------------------------------------
John Carmack's .plan for Jan 01, 1998
-----------------------------------------
Some of the things I have changed recently:
* fixed the cinematics
* don't clear config after dedicated server
* don't reallocate sockets unless needed
* don't process channel packets while connecting
* rate variable for modem bandwidth choking
* delta compress client usercmds
* fixed sound quality changing after intermissions
* fixed PVS problem when head was directly under solid in GL
* added r_drawflat and cl_testlights to cheats
There are a few problems that I am still trying to track down:
WSAEADDRNOTAVAIL errors
Map versions differ error
Sometimes connecting and seeing messages but not getting in
Decompression read overrun.
Of course, we don't actually get any of those errors on any of our systems here, so I am having to work remotely with other users to try and fix them, which is a bit tougher.
My new years resolution is to improve my coding style by bracing all single line statements and consistantly using following caps on multi word variable names.
Actually, I am currently trying on the full sun coding style, but I'm not so sure about some of the comment conventions: don't use multiple lines of // comments, and don't use rows of seperating characters in comments. I'm not convinced those are good guidelines.
-----------------------------------------
John Carmack's .plan for Jan 02, 1998
-----------------------------------------
Wired magazine does something that almost no other print magazine we have dealt with does.
They check the statements they are going to print.
I just got a "fact check" questionaire email from wired about an upcoming article, and I recall that they did this last time the did an article about us.
Most of the time when we talk with the press, we try to get them to send us a proof of the article for fact checking. They usually roll their eyes, and grudgingly agree, then don't send us anything, or send it to us after it has gone to press.
Wired had a few errors in their statements, but it won't get printed that way because they checked with us.
How refreshing.
--
A small public announcement:
The Linux Expo is looking for:
1. People that develop games or game servers in *nix, and 2. People interested in learning how to develop games in *nix.
Either one should give a write to ddt@crack.com.
-----------------------------------------
John Carmack's .plan for Jan 03, 1998
-----------------------------------------
New stuff fixed:
* timeout based non-active packet streams
* FS_Read with CD off checks
* dedicated server not allocate client ports
* qport proxy checking stuff
* fixed mouse wheel control
* forced newlines on several Cbuf_AddText ()
* if no nextmap on a level, just stay on same one
* chat maximums to prevent user forced overflows
* limit stringcmds per frame to prevent malicious use
* helped jumping down slopes
* checksum client move messages to prevent proxy bots
* challenge / response connection process
* fixed rcon
* made muzzle flash lights single frame, rather than 0.1 sec
I still don't have an answer to the WAADRNOTAVAILABLE problem. I have made the packet stream as friendly as possible, but some computers are still choking.
I managed to get fixes for address translating routers done without costing any bandwidth from the server, just a couple bytes from the client, which isn't usually a critical path.
I have spent a fair amount of time trying to protect against "bad" users in this release. I'm sure there will be more things that come up, but I know I got a few of the ones that are currently being exploited.
We will address any attack that can make a server crash. Other attacks will have to have the damage and prevelence weighed against the cost of defending against it.
Client message overflows. The maximum number of commands that can be issued in a user packet has been limited. This prevents a client from doing enough "says" or "kills" to overflow the message buffers of other clients.
Challenge on connection. A connection request to a server is now a two stage process of requesting a challenge, then using it to connect. This prevents denial of service attacks where connection packets with forged IPs are flooded at a server, preventing any other users from connecting until they timeout.
Client packet checksumming. The packets are encoded in a way that will prevent proxies that muck with the packet contents, like the stoogebot, from working.
-----------------------------------------
John Carmack's .plan for Jan 04, 1998
-----------------------------------------
Version 3.10 patch is now out.
ftp://ftp.idsoftware.com/idstuff/quake2/q2-310.exe
A few more minor fixes since yesterday:
* qhost support
* made qport more random
* fixed map reconnecting
* removed s_sounddir
* print out primary / secondary sound buffer status on init
* abort game after a single net error if not dedicated
* fixed sound loss when changing sound compatability
* removed redundant reliable overflow print on servers
* gl_lockpvs for map development checking
* made s_primary 0 the default
Christian will be updating the bug page tomorrow. So hold of on all reporting for 24 hours, then check the page to make sure the bug is not already known.
http://www.idsoftware.com/cgi-win/bugs.exe
All bug reports should go to Christian: xian@idsoftware.com.
I have had several cases of people with lockup problems and decompression overreads having their problems fixed after they mentioned that they were overclocking either their CPU, their system bus (to 75mhz), or their 3DFX.
It doesn't matter if "it works for everything else", it still may be the source of the problem.
I know that some people are still having problems with vanilla systems, though. I have tried everything I can think of remotely, but if someone from the Dallas area wants to bring a system by our office, I can try some more serious investigations.
Something that has shown to help with some 3dfx problems is to set "cl_maxfps 31", which will keep the console between level changes from rendering too fast, which has caused some cards to hang the system.
-----------------------------------------
John Carmack's .plan for Jan 09, 1998
-----------------------------------------
We got 70 people on a base100 server, and it died after it wedged at 100% utilization for a while. Tomorrow we will find exactly what overflowed, and do some profiling.
Base100 is really only good for 50 or so players without overcrowding, but we have another map being built that should hold 100 people reasonably well.
I will look into which will be the easier path to more server performance: scalar optimization of whatever is critical now, or splitting it off into some more threads to run on multiple processors. Neither one is trivial.
My goal is to be able to host stable 100 player games in a single map.
I just added a "players" command that will dump the total number of players in the game, and as many frags/names as it can fit in a packet (around 50, I think).
-----------------------------------------
John Carmack's .plan for Jan 11, 1998
-----------------------------------------
I AM GOING OUT OF TOWN NEXT WEEK, DON'T SEND ME ANY MAIL!
Odds are that I will get back and just flush the 500 messages in my mailbox.
No, I'm not taking a vacation. Quite the opposite, in fact.
I'm getting a hotel room in a state where I don't know anyone, so I can do a bunch of research with no distractions.
I bought a new computer specifically for this purpose - A Dolch portable pentium-II system. The significant thing is that it has full length PCI slots, so I was able to put an Evans & Sutherland OpenGL accelerator in it (not enough room for an intergraph Realizm, though), and still drive the internal LCD screen. It works out pretty well, but I'm sure there will be conventional laptops with good 3D acceleration available later this year.
This will be an interesting experiment for me. I have always wondered how much of my time that isn't at peak productivity is a necessary rest break, and how much of it is just wasted.
---
The client's IP address is now added to the userinfo before calling ClientConnect(), so any IP filtering / banning rules can now be implemented in the game dll. This will also give some of you crazy types the ability to sync up with multiple programs on the client computers outside of Q2 itself.
A new API entry point has been added to the game dll that gets called whenever an "sv" command is issued on the server console. This is to allow you to create commands for the server operator to type, as opposed to commands that a client would type (which are defined in g_cmds.c).
---
We did a bunch of profiling today, and finaly got the information I wanted. We weren't doing anything brain dead stupid in the server, and all of the time was pretty much where I expected it to be.
I did found two things we can pursue for optimization.
A moderately expensive catagorization function is called at both the beginning and end of client movement simulation. With some care, we should be able to avoid the first one most of the time. That alone should be good for a >10% server speedup.
The other major thing is that the client movement simulation accounted for 60% of the total execution time, and because it was already compartmentalized for client side prediction, it would not be much work to make it thread safe. Unfortunately, it would require MAJOR rework of the server code (and some of the game dll) to allow multiple client commands to run in parallel.
The potential is there to double the peak load that a server can carry if you have multiple processors. Note that you will definately get more players / system by just running multiple independent servers, rather than trying to get them all into a single large server.
We are not going to pursue either of these optimizations right now, but they will both be looked at again later.
All this optimizing of the single server is pushing the tail end of a paradigm. I expect trinity to be able to seamlessly hand off between clustered servers without the client even knowing it happened.
-----------------------------------------
John Carmack's .plan for Feb 04, 1998
-----------------------------------------
Ok, I'm overdue for an update.
The research getaway went well. In the space of a week, I only left my hotel to buy diet coke. It seems to have spoiled me a bit, the little distractions in the office grate on me a bit more since. I will likely make week long research excursions a fairly regular thing during non- crunch time. Once a quarter sounds about right.
I'm not ready to talk specifically about what I am working on for trinity. Quake went through many false starts (beam trees, portals, etc) before settling down on its final architecture, so I know that the odds are good that what I am doing now won't actually be used in the final product, and I don't want to mention anything that could be taken as an implied "promise" by some people.
I'm very excited by all the prospects, though.
Many game developers are in it only for the final product, and the process is just what they have to go through to get there. I respect that, but my motivation is a bit different.
For me, while I do take a lot of pride in shipping a great product, the achievements along the way are more memorable. I don't remember any of our older product releases, but I remember the important insights all the way back to using CRTC wraparound for infinate smooth scrolling in Keen (actually, all the way back to understanding the virtues of structures over parallel arrays in apple II assembly language..). Knowledge builds on knowledge.
I wind up catagorizing periods of my life by how rich my learning experiences were at the time.
My basic skills built up during school on apple II computers, but lack of resources limited how far and fast I could go. The situation is so much better for programmers today - a cheap used PC, a linux CD, and an internet account, and you have all the tools and resources necessary to work your way to any level of programming skill you want to shoot for.
My first six months at Softdisk, working on the PC, was an incredible learning experience. For the first time, I was around a couple of programmers with more experience than I had (Romero and Lane Roath), there were a lot of books and materials available, and I could devote my full and undivided attention to programming. I had a great time.
The two years following, culminating in DOOM and the various video game console work I did, was a steady increase in skills and knowledge along several fronts - more graphics, networking, unix, compiler writing, cross development, risc architectures, etc.
The first year of Quake's development was awesome. I got to try so many new things, and I had Michael Abrash as my sounding board. It would probably surprise many classically trained graphics programmers how little I new about conventional 3D when I wrote DOOM - hell, I had problems properly clipping wall polygons (which is where all the polar coordinate nonsense came from). Quake forced me to learn things right, as well as find some new innovations.
The last six months of Quake's development was mostly pain and suffering trying to get the damn thing finished. It was all worth it in the end, but I don't look back at it all that fondly.
The development cycle of Quake 2 had some moderate learning experiences for me (glquake, quakeworld, radiosity, openGL tool programming, win32, etc), but it also gave my mind time to sift through a lot of things before getting ready to really push ahead.
I think that the upcoming development cycle for trinity is going to be at least as rewarding as Quake's was. I am reaching deep levels of understanding on some topics, and I am branching out into several completely new (non-graphics) areas for me, that should cross-polinate well with everything else I am doing.
There should also be a killer game at the end of it. :)
-----------------------------------------
John Carmack's .plan for Feb 09, 1998
-----------------------------------------
Just got back from the Q2 wrap party in vegas that Activision threw for us.
Having a reasonable grounding in statistics and probability and no belief in luck, fate, karma, or god(s), the only casino game that interests me is blackjack.
Playing blackjack properly is a test of personal discipline. It takes a small amount of skill to know the right plays and count the cards, but the hard part is making yourself consistently behave like a robot, rather than succumbing to your "gut instincts".
I play a basic high/low count, but I scale my bets widely - up to 20 to 1 in some cases. Its not like I'm trying to make a living at it, so the chance of getting kicked out doesn't bother me too much.
I won $20,000 at the tables, which I am donating to the Free Software Foundation. I have been meaning to do something for the FSF for a long time. Quake was deployed on a dos port of FSF software, and both DOOM and Quake were developed on NEXTSTEP, which uses many FSF based tools. I don't subscribe to all the FSF dogma, but I have clearly benefited from their efforts.
-----------------------------------------
John Carmack's .plan for Feb 12, 1998
-----------------------------------------
I have been getting a lot of mail with questions about the intel i740 today, so here is a general update on the state of 3D cards as they relate to quake engine games.
ATI rage pro
----
On paper, this chip looks like it should run almost decently - about the performance of a permedia II, but with per-pixel mip mapping and colored lighting. With the currently shipping MCD GL driver on NT, it just doesn't run well at all. The performance is well below acceptable, and there are some strange mip map selection errors. We have been hearing for quite some time that ATI is working on an OpenGL ICD for both '95 and NT, but we haven't seen it yet. The rage pro supposedly has multitexture capability, which would help out quite a bit if they implement the multitexture extension. If they do a very good driver, the rage pro may get up to the performance of the rendition cards. Supports up to 16MB, which would make it good for development work if the rest of it was up to par.
3DLabs permedia II
------
Good throughput, poor fillrate, fair quality, fair features.
No colored lighting blend mode, currently no mip mapping at all.
Supports up to 8MB.
The only currently shipping production full ICD for '95, but a little flaky.
If 3dlabs implemented per-polygon mip mapping, they would get both a quality and a slight fillrate boost.
Drivers available for WinNT on the DEC Alpha (but the alpha drivers are very flaky).
Power VR PCX2
-----
Poor throughput, good fillrate, fair quality, poor features, low price.
No WinNT support.
Almost no blend modes at all, low alpha precision.
Even though the hardware doesn't support multitexture, they could implement the multi-texture extension just to save on polygon setup costs. That might get them a 10% to 15% performance boost.
They could implement the point parameters extension for a significant boost in the speed of particle rendering. That wouldn't affect benchmark scores very much, but it would help out in hectic deathmatches.
Their opengl minidriver is already a fairly heroic effort - the current PVR takes a lot of beating about the head to make it act like an OpenGL accelerator.
Rendition v2100 / v2200
--------
Good throughput, good fillrate, very good quality, good features.
A good all around chip. Not quite voodoo1 performance, but close.
v2100 is simply better than everything else in the $99 price range.
Can render 24 bit color for the best possible quality, but their current drivers don't support it. Future ones probably will.
Can do 3D on the desktop.
Rendition should be shipping a full ICD OpenGL, which will make an 8mb v2200 a very good board for people doing 3D development work.
NVidia Riva 128
-----
Very good throughput, very good fillrate, fair quality, fair features.
The fastest fill rate currently shipping, but it varies quite a bit based on texture size. On large textures it is slightly slower than voodoo, but on smaller textures it is over twice as fast.
On paper, their triangle throughput rate should be three times what voodoo gives, but in practice we are only seeing a slight advantage on very fast machines, and worse performance on pentium class machines. They probably have a lot of room to improve that in their drivers.
In general, it is fair to say that riva is somewhat faster than voodoo 1, but it has a few strikes against it.
The feature implementation is not complete. They have the blend mode for colored lighting, but they still don't have them all. That may hurt them in future games. Textures can only be 1 to 1 aspect ratio. In practice, that just means that non-square textures waste memory.
The rendering quality isn't quite as high as voodoo or rendition. It looks like some of their iterators don't have enough precision.
Nvidia is serious and committed to OpenGL. I am confident that their driver will continue to improve in both performance and robustness.
While they can do good 3D in a window, they are limited to a max of 4MB of framebuffer, which means that they can't run at a high enough resolution to do serious work.
3DFX Voodoo 1
-----
The benchmark against which everything else is measured.
Good throughput, good fillrate, good quality, good features.
It has a couple faults, but damn few: max texture size limited to 256*256 and 8 to 1 aspect ratio. Slow texture swapping. No 24 bit rendering.
Because of the slow texture swapping, anyone buying a voodoo should get a six mb board (e.g. Canopus Pure3D). The extra ram prevents some sizable jerks when textures need to be swapped.
Highly tuned minidriver. They have a full ICD in alpha, but they are being slow about moving it into production. Because of the add-in board nature of the 3dfx, the ICD won't be useful for things like running level editors, but it would at least guarantee that any new features added to quake engine games won't require revving the minidriver to add new functionality.
3DFX Voodoo 2
-----
Not shipping yet, but we were given permission to talk about the benchmarks on their preproduction boards.
Excellent throughput, excellent fillrate, good quality, excellent features.
The numbers were far and away the best ever recorded, and they are going to get significantly better. On quake 2, voodoo 2 is setup limited, not fill rate limited. Voodoo 2 can do triangle strip and fan setup in hardware, but their opengl can't take advantage of it until the next revision of glide. When that happens, the number of vertexes being sent to the card will drop by HALF. At 640*480, they will probably become fill rate bound again (unless you interleave two boards), but at 512*384, they will probably exceed 100 fps on a timedemo. In practice, that means that you will play the game at 60 fps with hardly ever a dropped frame.
The texture swapping rate is greatly improved, addressing the only significant problem with voodoo.
I expect that for games that heavily use multitexture (all quake engine games), voodoo 2 will remain the highest performer for all of '98. All you other chip companies, feel free to prove me wrong. :)
Lack of 24 bit rendering is the only visual negative.
As with any voodoo solution, you also give up the ability to run 3D applications on your desktop. For pure gamers, that isn't an issue, but for hobbyists that may be interested in using 3D tools it may have some weight.
Intel i740
----
Good throughput, good fillrate, good quality, good features.
A very competent chip. I wish intel great success with the 740. I think that it firmly establishes the baseline that other companies (especially the ones that didn't even make this list) will be forced to come up to.
Voodoo rendering quality, better than voodoo1 performance, good 3D on a desktop integration, and all textures come from AGP memory so there is no texture swapping at all.
Lack of 24 bit rendering is the only negative of any kind I can think of.
Their current MCD OpenGL on NT runs quake 2 pretty well. I have seen their ICD driver on '95 running quake 2, and it seems to be progressing well. The chip has the potential to outperform voodoo 1 across the board, but 3DFX has more highly tuned drivers right now, giving it a performance edge. I expect intel will get the performance up before releasing the ICD.
It is worth mentioning that of all the drivers we have tested, intel's MCD was the only driver that did absolutely everything flawlessly. I hope that their ICD has a similar level of quality (it's a MUCH bigger job).
An 8mb i740 will be a very good setup for 3D development work.
-----------------------------------------
John Carmack's .plan for Feb 16, 1998
-----------------------------------------
8 mb or 12 mb voodoo 2?
An 8mb v2 has 2 mb of texture memory on each TMU. That is not as general as the current 6mb v1 cards that have 4 mb of texture memory on a single TMU. To use the multitexture capability, textures are restricted to being on one or the other TMU (simplifying a bit here). There is some benefit over only having 2 mb of memory, but it isn't double. You will see more texture swapping in quake on an 8mb voodoo 2 than you would on a 6mb voodoo 1. However, the texture swapping is several times faster, so it isn't necessarily all that bad.
If you use the 8 bit palettized textures, there will probably not be any noticable speed improvement with a 12 mb voodoo 2 vs an 8 mb one. The situation that would most stress it would be an active deathmatch that had players using every skin. You might see a difference there.
A game that uses multitexture and 16 bit textures for everything will stress a 4/2/2 voodoo layout. Several of the Quake engine licensees are using full 16 bit textures, and should perform better on a 4/4/4 card.
The differences probably won't show as significant on timedemo numbers, but they will be felt as little one frame hitches here and there.
-----------------------------------------
John Carmack's .plan for Feb 17, 1998
-----------------------------------------
I just read the Wired article about all the Doom spawn.
I was quoted as saying "like I'm supposed to be scared of Monolith", which is much more derogatory sounding than I would like.
I haven't followed Monolith's development, and I don't know any of their technical credentials, so I am not in any position to evaluate them.
The topic of "is microsoft going to crush you now that they are in the game biz", made me a bit sarcastic.
I honestly wish the best to everyone pursuing new engine development.
-----------------------------------------
John Carmack's .plan for Feb 22, 1998
-----------------------------------------
Don't send any bug reports on the 3.12 release to me, I just forward them over to jcash. He is going to be managing all future work on the Quake 2 codebase through the mission packs. I'm working on trinity.
3.12 answered the release question pretty decisively for me. We were in code freeze for over two weeks while the release was being professionally beta tested, and all it seemed to get us was a two week later release.
Future releases are going to be of the fast/multiple release type, but clearly labeled as a "beta" release until it stabilizes. A dozen professional testers or fifty amature testers just can't compare to the thousands of players who will download a beta on the first day.
I have spent a while thinking about the causes of the patches for Q2. Our original plan was to just have the contents of 3.12 as the first patch, but have it out a month earlier than we did.
The first several patches were forced due to security weaknesses. Lesson learned - we need to design more security conscious to try to protect against the assholes out there.
The cause for the upcoming 3.13 patch is the same thing that has caused us a fair amount of trouble through Q2's development - instability in the gamex86 code due to its decending from QC code in Q1. It turns out that there were lots of bugs in the original QC code, but because of its safe interpreted nature (specifically having a null entity reference the world) they never really bothered anyone. We basically just ported the QC code to regular C for Q2 (it shows in the code) and fixed crash bugs as they popped up. We should have taken the time to redesign more for C's strengths and weaknesses.
-----------------------------------------
John Carmack's .plan for Mar 12, 1998
-----------------------------------------
American McGee has been let go from Id.
His past contributions include work in three of the all time great games (DOOM 2, Quake, Quake 2), but we were not seeing what we wanted.
-----------------------------------------
John Carmack's .plan for Mar 13, 1998
-----------------------------------------
The Old Plan:
The rest of the team works on an aggressive Quake 2 expansion pack while Brian and I develop tools and code for the entirely new Trinity generation project to begin after the mission pack ships.
The New Plan:
Expand the mission pack into a complete game, and merge together a completely new graphics engine with the quake 2 game / client / server framework, giving us Quake 3.
"Trinity" is basically being broken up into two phases: graphics and everything else. Towards the end of Quake 1's development I was thinking that we might have been better off splitting quake on those categories, but in reverse order. Doing client/server, the better modification framework, and qc, coupled with a spiced up DOOM engine (Duke style) for one game, then doing the full 3D renderer for the following game.
We have no reason to believe that the next generation development would somehow go faster than the previous, so there is a real chance that doing all of the Trinity technology at once might push game development time to a full two years for us, which might be a bit more than the pressure-cooker work atmosphere here could handle.
So, we are going to try an experiment.
The non-graphics things that I was planning for Trinity will be held off until the following project - much java integration with client downloadable code being one of the more significant aspects. I hope to get to some next generation sound work, but the graphics engine is the only thing I am committing to.
The graphics engine is going to be hardware accelerated ONLY. NO SOFTWARE RENDERER, and it won't work very well on a lot of current hardware. We understand fully that this is going to significantly cut into our potential customer base, but everyone was tired of working under the constraints of the software renderer. There are still going to be plenty of good quake derived games to play from other developers for people without appropriate hardware.
There are some specific things that the graphics technology is leveraging that may influence your choice of a 3D accelerator.
All source artwork is being created and delivered in 24 bit color. An accelerator that can perform all 3D work in 24 bit color will look substantially better than a 16 bit card. You will pay a speed cost for it, though.
Most of the textures are going to be higher resolution. Larger amounts of texture memory will make a bigger difference than it does on Quake 2.
Some key rendering effects require blending modes that some cards don't support.
The fill rate requirements will be about 50% more than Quake 2, on average. Cards that are fill rate limited will slow down unless you go to a lower resolution.
The triangle rate requirements will be at least double Quake 2, and scalable to much higher levels of detail on appropriate hardware.
Here are my current GUESSES about how existing cards will perform.
Voodoo 1 Performance will be a little slow, but it should look good and run acceptably. You will have to use somewhat condensed textures to avoid texture thrashing.
Voodoo 2 Should run great. Getting the 12 mb board is probably a good idea if you want to use the high resolution textures. The main rendering mode won't be able to take advantage of the dual TMU the same way quake 2 does, so the extra TMU will be used for slightly higher quality rendering modes instead of greater speed: trilinear / detail texturing, or some full color effects where others get a mono channel.
Permedia 2 Will be completely fill rate bound, so it will basically run 2/3 the speed that quake 2 does. Not very fast. It also doesn't have one of the needed blending modes, so it won't look very good, either. P2 does support 24 bit rendering, but it won't be fast enough to use it.
ATI Rage Pro It looks like the rage pro has all the required blending modes, but the jury is still out on the performance.
Intel I740 Should run good with all features, and because all of the textures come out of AGP memory, there will be no texture thrashing at all, even with the full resolution textures.
Rendition 2100/2200 The 2100 should run about the speed of a voodoo 1, and the 2200 should be faster. They support all the necessary features, and an 8 mb 2200 should be able to use the high res textures without a problem. The renditions are the only current boards that can do 24 bit rendering with all the features. It will be a bit slow in 24 bit mode, but it will look the best.
PVR PCX2 Probably won't run Quake 3. They don't have ANY of the necessary blending modes, so it can't look correct. Video Logic might decide to rev their minidriver to try to support it, but it is probably futile.
RIVA 128 Riva puts us in a bad position. They are very fast, but they don't support an important feature. We can crutch it up by performing some extra drawing passes, but there is a bit of a quality loss, and it will impact their speed. They will probably be a bit faster than voodoo 1, but not to the degree that they are in Quake 2.
Naturally, the best cards are yet to come (I won't comment on unreleased cards). The graphics engine is being designed to be scalable over the next few YEARS, so it might look like we are shooting a bit high for the first release, but by the time it actually ships, there will be a lot of people with brand new accelerators that won't be properly exploited by any other game.
-----------------------------------------
John Carmack's .plan for Mar 20, 1998
-----------------------------------------
Robert Duffy, the maintainer of Radiant QE4 is now "officially" in charge of further development of the editor codebase. He joins Zoid as a (part time) contractor for us.
A modified version of Radiant will be the level editor for Quake 3. The primary changes will be support for curved surfaces and more general surface shaders. All changes will be publicly released, either after Q3 ships or possibly at the release of Q3Test, depending on how things are going.
The other major effort is to get Radiant working properly on all of the 3D cards that are fielding full OpenGL ICDs. If you want to do level development, you should probably get an 8mb video card. Permedia II cards have been the mainstay for developers that can't afford intergraph systems, but 8mb rendition v2200 (thriller 3D) cards are probably a better bet as soon as their ICD gets all the bugs worked out.
-----------------------------------------
John Carmack's .plan for Mar 21, 1998
-----------------------------------------
I just shut down the last of the NEXTSTEP systems running at id.
We hadn't really used them for much of anything in the past year, so it was just easier to turn them off than to continue to administer them.
Most of the intel systems had already been converted to NT or 95, and Onethumb gets all of our old black NeXT hardware, but we have four nice HP 712/80 workstations that can't be used for much of anything.
If someone can put these systems to good use (a dallas area unix hacker), you can have them for free. As soon as they are spoken for, I will update my .plan, so check immediately before sending me email.
You have to come by our office (in Mesquite) and do a fresh OS install here before you can take one. There may still be junk on the HD, and I can't spend the time to clean them myself. You can run either NEXTSTEP 3.3 or HP/UX. These are NOT intel machines, so you can't run dos or windows. I have NS CD's here, but I can't find the original HP/UX CDs. Bring your own if that's what you want.
I'm a bit nostalgic about the NeXT systems -- the story in the Id Anthology is absolutely true: I walked through a mile of snow to the bank to pay for our first workstation. For several years, I considered it the best development environment around. It still has advantages today, but you can't do any accelerated 3D work on it.
I had high hopes for rhapsody, but even on a top of the line PPC, it felt painfully sluggish compared to the NT workstations I use normally, and apple doesn't have their 3D act together at all.
Its kind of funny, but even through all the D3D/OpenGL animosity, I think Windows NT is the best place to do 3D graphics development.
All gone!
--------------
Paul Magyar gets the last (slightly broken) one.
Bob Farmer gets the third.
Philip Kizer gets the second one.
Kyle Bousquet gets the first one.
3/21 pt 2
---------
I haven't given up on rhapsody yet. I will certainly be experimenting with the release version when it ships, but I have had a number of discouraging things happen. Twice I was going to go do meetings at apple with all relevent people, but the people setting it up would get laid off before the meetings happened. Several times I would hear encouraging rumors about various things, but they never panned out. We had some biz discussions with apple about rhapsody, but they were so incredibly cautious about targeting rhapsody for consumer apps at the expense of macos that I doubted their resolve.
I WANT to help. Maybe post-E3 we can put something together.
The SGI/microsoft deal fucked up a lot of the 3D options. The codebase that everyone was using to develop OpenGL ICDs is now owned by microsoft, so it is unlikely any of them will ever be allowed to port to rhapsody (or linux, or BeOS).
That is one of the things I stress over -- The Right Thing is clear, but its not going to happen because of biz moves. It would be great if ATI, which has video drivers for win, rhapsody, linux, and BeOS, could run the same ICD on all those platforms.
-----------------------------------------
John Carmack's .plan for Mar 26, 1998
-----------------------------------------
I haven't even seen the "BeOS port of Quake". Stop emailing me about aproving it. I told one of the Lion developers he could port it to BeOS in his spare time, but I haven't seen any results from it.
-
There is a public discussion / compilation going on at openquake for suggestions to improve technical aspects of quake 3:
http://www.openquake.org/q3suggest/
This is sooo much better than just dropping me an email when a thought hits you. There are many, many thousands of you out there, and there needs to be some filtering process so we can get the information efficiently.
We will read and evaluate everything that makes it through the discussion process. There are two possible reasons why features don't make it into our games - either we decide that the effort is better spent elsewhere, or we just don't think about it. Sometimes the great ideas are completely obvious when suggested, but were just missed. That is what I most hope to see.
When the suggestions involve engineering tradeoffs and we have to consider the implementation effort of a feature vs its benefits, the best way to convince us to pursue it is to specify EXACTLY what benefits would be gained by undertaking the work, and specifying a clean interface to the feature from the file system data and the gamex86 code.
We hack where necessary, but I am much more willing to spend my time on an elegant extension that has multiple uses, rather than adding api bulk for specific features. Point out things that are clunky and inelegant in the current implementation. Even if it doesn't make any user visible difference, restructuring api for cleanliness is still a worthwhile goal.
We have our own ideas about game play features, so we may just disagree with you. Even if you-and-all-your-friends are SURE that your suggestions will make the game a ton better, we may not think it fits with our overall direction. We aren't going to be all things to all people, and we don't design by committee.
-----------------------------------------
John Carmack's .plan for Apr 02, 1998
-----------------------------------------
Drag strip day!
Most of the id guys, John Romero from ION, and George and Alan from 3drealms headed to the Ennis dragstrip today.
Nobody broke down, and some good times were posted.
11.9 @ 122 John Carmack F40
12.2 @ 122 George Broussard custom turbo 911
12.4 @ 116 Brian Hook Viper GTS
13.4 @ 106 John Romero custom turbo testarossa
13.6 @ 106 Todd Hollenshead 'vette
13.9 @ 100 Paul Steed 911
14.0 @ 99 Tim Willits 911
14.3 @ 101 Bear Turbo Supra
14.4 @ 98 Alan Blum turbo rx-7
14.7 @ 92 Brandon James M3
15.3 @ 92 Christian Boxster
15.5 @ 93 Jen (Hook's Chick) Turbo Volvo
16.1 @ 89 Ms. Donna Mustang GT
17.4 @ 82 Anna (Carmack's Chick) Honda Accord
18.1 @ 75 Jennifer (Jim Molinets' Chick) Saturn
We had three significant no-shows for various reasons: my TR, Adrian's viper, and Cash's supercharged M3 were all in the shop.
-----------------------------------------
John Carmack's .plan for Apr 08, 1998
-----------------------------------------
Things are progressing reasonably well on the Quake 3 engine.
Not being limited to supporting a 320*240 256 color screen is very, very nice, and will make everyone's lives a lot easier.
All of our new source artwork is being done in 24 bit TGA files, but the engine will continue to load .wal files and .pcx files for developer's convenience. Each pcx can have its own palette now though, because it is just converted to 24 bit at load time.
Q3 is going to have a fixed virtual screen coordinate system, independant of resolution. I tried that back in the original glquake, but the fixed coordinate system was only 320*200, which was excessively low. Q2 went with a dynamic layout at different resolutions, which was a pain, and won't scale to the high resolutions that very fast cards will be capable of running at next year.
All screen drawing is now done assuming the screen is 640*480, and everything is just scaled as you go higher or lower. This makes laying out status bars and HUDs a ton easier, and will let us do a lot cooler looking screens.
There will be an interface to let game dlls draw whatever they want on the screen, precisely where they want it. You can suck up a lot of network bandwidth doing that though, so some care will be needed.
-
Going to the completely opposite end of the hardware spectrum from quake 3..
I have been very pleased with the fallout from the release of the DOOM source code.
At any given spot in design space, there are different paths you can take to move forward. I have usually chosen to try to make a large step to a completely new area, but the temptation is there to just clean up and improve in the same area, continuously polishing the same program.
I am enjoying seeing several groups pouring over DOOM, fixing it up and enhancing it. Cleaning up long standing bugs. Removing internal limitations. Orthogonalizing feature sets. Etc.
The two that I have been following closest are Team TNT's BOOM engine project, which is a clear headed, well engineered improvement on the basic DOOM technical decisions, and Bruce Lewis' glDoom project.
Any quakers feeling nostalgic should browse around:
http://www.doomworld.com/
-----------------------------------------
John Carmack's .plan for Apr 16, 1998
-----------------------------------------
F40 + $465,000 = F50
-----------------------------------------
John Carmack's .plan for Apr 17, 1998
-----------------------------------------
Yes, I bought an F50. No, I don't want a McLaren.
We will be going back to the dragstrip in a couple weeks, and I will be exercising both the F50 and the TR there. Cash's supercharged M3 will probably show some of the porsches a thing or two, as well.
I'll probably rent a road coarse sometime soon, but I'm not in too much of a hurry to run the F50 into the weeds.
My TR finally got put back together after a terrific nitrous explosion just before the last dragstrip. It now makes 1000.0 hp at the rear wheels. Contrast that with the 415 rear wheel hp that the F40 made. Of course, a loaded testarossa does weigh about 4000 lbs..
My project car is somewhat nearing completion. My mechanic says it will be running in six weeks, but mechanics can be even more optimistic than software developers. :) I'm betting on fall. It should really be something when completed: a carbon fiber bodied ferrari GTO with a custom, one-of-a kind billet alluminum 4 valve DOHC 5.2L V12 with twin turbos running around 30 lbs of boost. It should be good for quite a bit more hp than my TR, and the entire car will only weigh 2400 lbs.
---
The distance between a cool demo and production code is vast. Two months ago, I had some functional demos of several pieces of the Quake 3 rendering tech, but today it still isn't usable as a full replacement for ref_gl yet.
Writing a modern game engine is a lot of work.
The new architecture is turning out very elegent. Not having to support software rendering or color index images is helping a lot, but it is also nice to reflect on just how much I have learned in the couple years since the original Quake renderer was written.
My C coding style has changed for Quake 3, which is going to give me a nice way of telling at a glance which code I have or haven't touched since Quake 2. In fact, there have been enough evolutions in my style that you can usually tell what year I wrote a piece of code by just looking at a single function:
[code]
/*
=============
=
= Function headers like this are DOOM or earlier
=
=============
*/
/*
=============
Function Headers like this are Quake or later
=============
*/
{
// comments not indented were written on NEXTSTEP
// (quake 1)
// indented comments were written on
// Visual C++ (glquake / quakeworld, quake2)
}
for (testnum=0 ; testnum<4 ; testnum++)
{ // older coding style
}
for (testNumber = 0 ; testNumber < 4 ; testNumber++) {
// quake 3 coding style
}
[/code]
-----------------------------------------
John Carmack's .plan for Apr 22, 1998
-----------------------------------------
F50 pros and cons vs F40:
The front and rear views are definately cooler on the F50, but I think I like the F40 side view better. I haven't taken the top off the F50 yet, though (its supposed to be a 40 minute job..).
Adjustable front suspension. Press a button and it raises two inches, which means you can actually drive it up into strip malls. The F40 had to be driven into my garage at an angle to keep the front from rubbing. This makes the car actually fairly practical for daily driving.
Drastically better off idle torque. You have to rev the F40 a fair amount to even get it moving, and if you are moving at 2000 rpm in first gear, a honda can pull away from you until it starts making boost at 3500 rpm. The f50 has enough torque that you don't even need to rev to get moving, and it goes quite well by just flooring it after you are moving. No need to wreck a clutch by slipping it out from 4000 rpm.
Much nicer clutch. The F40 clutch was a very low-tech single disk clutch that required more effort than on my crazy TR with over twice the torque.
Better rearward visibility. The F40's lexan fastback made everything to your rear a blur.
Better shifting. A much smoother six speed than the F40's five speed.
Better suspension. Some bumps that would upset the F40 badly are handled without any problems.
Better aerodynamics. A flat underbody with tunnels is a good thing if you are going to be moving at very high speeds.
I beleive the F50 could probably lap a road coarse faster than the F40, but in a straight line, the F40 is faster. The F50 felt a fair amount slower, but I was chalking that up to the lack of non-linear turbo rush. Today I drove it down to the dyno and we got real numbers.
It only made 385 hp at the rear wheels, which is maybe 450 at the crank if you are being generous. The F40 made 415, but that was with the boost cranked up a bit over stock.
We're going to have to do something about that.
I'm thinking that a mild twin-turbo job will do the trick. Six pounds of boost should get it up to a health 500 hp at the rear wheels, which will keep me happy. I don't want to turn it into a science project like my TR, I just want to make sure it is well out of the range of any normal cars.
I may put that in line after my GTO gets finished.
-----------------------------------------
John Carmack's .plan for May 02, 1998
-----------------------------------------
The rcon backdoor was added to help the development of QuakeWorld (It is not present in Quake 1). At the time, attacking Quake servers with spoofed packets was not the popular sport it seems to have become with Quake 2, so I didn't think much about the potential for exploitation.
The many forced releases of Quake 2 due to hacker attacks has certainly taught me to be a lot more wary.
It was a convenient feature for us, but it turned out to be irresponsible. Sorry.
There will be new releases of QuakeWorld and Quake 2 soon.
-----------------------------------------
John Carmack's .plan for May 04, 1998
-----------------------------------------
Here are some notes on a few of the technologies that I researched in preparing for the Quake3/trinity engine. I got a couple months of pretty much wide open research done at the start, but it turned out that none of the early research actually had any bearing on the directions I finally decided on. Ah well, I learned a lot, and it will probably pay off at some later time.
I spent a little while doing some basic research with lummigraphs, which are sort of a digital hologram. The space requirements are IMMENSE, on the order of several gigs uncompressed for even a single full sized room. I was considering the possibility of using very small lumigraph fragments (I called them "lumigraphlets") as imposters for large clusters of areas, similar to aproximating an area with a texture map, but it would effectively be a view dependent texture.
The results were interesting, but transitioning seamlessly would be difficult, the memory was still large, and it has all the same caching issues that any impostor scheme has.
Another aproach I worked on was basically extending the sky box code style of rendering from quake 2 into a complete rendering system. Take a large number of environment map snapshots, and render a view by interpolating between up to four maps (if in a tetrahedral arangement) based on the view position.
A simple image based interpolating doesn't convey a sense of motion, because it basically just ghosts between seperate points unless the maps are VERY close together reletive to the nearest point visible in the images.
If the images that make up the environment map cube also contain depth values at some (generally lower) resolution, instead of rendering the environment map as six big flat squares at infinity, you can render it as a lot of little triangles at the proper world coordinates for the individual texture points. A single environment map like this can be walked around in and gives a sense of motion. If you have multiple maps from nearby locations, they can be easily blended together. Some effort should be made to nudge the mesh samples so that as many points are common between the maps as possible, but even a regular grid works ok.
You get texture smearing when occluded detail should be revealed, and if you move too far from the original camera point the textures blur out a lot, but it is still a very good effect, is completely complexity insensitive, and is aliasing free except when the view position causes a silhouette crease in the depth data.
Even with low res environment maps like in Quake2, each snapshot would consume 700k, so taking several hundred environment images throughout a level would generate too much data. Obviously there is a great deal of redundancy - you will have several environment maps that contain the same wall image, for instance. I had an interesting idea for compressing it all. If you ignore specular lighting and atmospheric effects, any surface that is visible in multiple environment maps can be represented by a single copy of it and perspective transformation of that image. Single image, transformations, sounds like.. fractal compression. Normal fractal compression only deals with affine maps, but the extension to projective maps seems logical.
I think that a certain type of game could be done with a technology like that, but in the end, I didn't think it was the right direction for a first person shooter.
There is a tie in between lummigraphs, multiple environment maps, specularity, convolution, and dynamic indirect lighting. Its nagging at me, but it hasn't come completely clear.
Other topics for when I get the time to write more:
Micro environment map based model lighting. Convolutions of environment maps by phong exponent, exponent of one with normal vector is diffuse lighting.
Full surface texture representation. Interior antaliasing with edge matched texels.
Octree represented surface voxels. Drawing and tracing.
Bump mapping, and why most of the aproaches being suggested for hardware are bogus.
Parametric patches vs implicit functions vs subdivision surfaces.
Why all analytical boundary representations basically suck.
Finite element radiosity vs photon tracing.
etc.
-----------------------------------------
John Carmack's .plan for May 17, 1998
-----------------------------------------
Here is an example of some bad programming in quake:
There are three places where text input is handled in the game: the console, the chat line, and the menu fields. They all used completely different code to manage the input line and display the output. Some allowed pasting from the system clipboard, some allowed scrolling, some accepted unix control character commands, etc. A big mess.
Quake 3 will finally have full support for international keyboards and character sets. This turned out to be a bit more trouble than expected because of the way Quake treated keys and characters, and it led to a rewrite of a lot of the keyboard handling, including the full cleanup and improvement of text fields.
A similar cleanup of the text printing hapened when Cash implemented general colored text: we had at least a half dozen different little loops to print strings with slightly different attributes, but now we have a generalized one that handles embedded color commands or force-to-color printing.
Amidst all the high end graphics work, sometimes it is nice to just fix up something elementary.
-----------------------------------------
John Carmack's .plan for May 19, 1998
-----------------------------------------
A 94 degree day at the dragstrip today. Several 3drealms and Norwood Autocraft folk also showed up to run. We got to weigh most of the cars on the track scales, which gives us a few more data points.
11.6 @ 125 Bob Norwood's ferrari P4 race car (2200 lbs)
11.9 @ 139 John Carmack's twin turbo testarossa (3815 lbs)
11.9 @ 117 Paul Steed's YZF600R bike
12.1 @ 122 John Carmack's F50 (3205 lbs)
12.3 @ 117 Brian's Viper GTS (3560 lbs)
13.7 @ 103 John Cash's supercharged M3
14.0 @ 96 Scott Miller's lexus GS400
15.0 @ ??? Someone's volkswagon GTI
15.1 @ ??? Christian's boxter (with Tim driving)
Weight is the key for good ETs. The TR has considerably better power to weight ratio than the P4, but it can't effectively use most of the power until it gets into third gear. The viper is actually making more power than the F50, (Brian got a big kick out of that after his dyno run) but 350 lbs more than compensated for it.
I wanted to hit 140 in the TR, but the clutch started slipping on the last run and I called it a day.
I was actually surprised the F50 ran 122 mph, which is the same the F40 did on a 25 degree cooler day. I was running with the top off, so it might even be capable of going a bit faster with it on.
The F50 and the viper were both very consistant performers, but the TR and the supercharged M3 were all over the place with their runs.
Brian nocked over a tenth off of his times even in spite of the heat, due to launch practice and some inlet modifications. He also power shifted on his best run.
It was pretty funny watching the little volkswagon consistantly beat up on a tire shredding trans-am.
George Broussard had his newly hopped up 911 turbo, but it broke the trans on its very first run. We were expecting him to be in the 11's.
We probably won't run again until either I get the F50 souped up, or my GTO gets finished.
-----------------------------------------
John Carmack's .plan for May 22, 1998
-----------------------------------------
Congratulations to Epic, Unreal looks very good.
-----------------------------------------
John Carmack's .plan for Jun 08, 1998
-----------------------------------------
I spent quite a while investigating the limits of input under windows recently. I foudn out a few interesting things:
Mouse sampling on win95 only happens every 25ms. It doesn't matter if you check the cursor or use DirectInput, the values will only change 40 times a second.
This means that with normal checking, the mouse control will feel slightly stuttery whenever the framerate is over 20 fps, because on some frames you will be getting one input sample, and on other frames you will be getting two. The difference between two samples and three isn't very noticable, so it isn't much of an issue below 20 fps. Above 40 fps it is a HUGE issue, because the frames will be bobbing between one sample and zero samples.
I knew there were some sampling quantization issues early on, so I added the "m_filter 1" variable, but it really wasn't an optimal solution. It averaged together the samples collected at the last two frames, which worked out ok if the framerate stayed consistantly high and you were only averaging together one to three samples, but when the framerate dropped to 10 fps or so, you wound up averaging together a dozen more samples than were really needed, giving the "rubber stick" feel to the mouse control.
I now have three modes of mouse control:
in_mouse 1: Mouse control with standard win-32 cursor calls, just like Quake 2.
in_mouse 2: Mouse control using DirectInput to sample the mouse relative counters each frame. This behaves like winquake with -dinput. There isn't a lot of difference between this and 1, but you get a little more precision, and you never run into window clamping issues. If at some point in the future microsoft changes the implementation of DirectInput so that it processes all pending mouse events exactly when the getState call happens, this will be the ideal input mode.
in_mouse 3: Processes DirectInput mouse movement events, and filters the amount of movement over the next 25 milliseconds. This effectively adds about 12 ms of latency to the mouse control, but the movement is smooth and consistant at any variable frame rate. This will be the default for Quake 3, but some people may want the 12ms faster (but rougher) response time of mode 2.
It takes a pretty intense player to even notice the difference in most cases, but if you have a setup that can run a very consistant 30 fps you will probably apreciate the smoothness. At 60 fps, anyone can tell the difference, but rendering speeds will tend to cause a fair amount of jitter at those rates no matter what the mouse is doing.
DirectInput on WindowsNT does not log mouse events as they happen, but seems to just do a poll when called, so they can't be filtered properly.
Keyboard sampling appears to be millisecond precise on both OS, though.
In doing this testing, it has become a little bit more tempting to try to put in more leveling optimizations to allow 60 hz framerates on the highest end hardware, but I have always shied away from targeting very high framerates as a goal, because when you miss by a tiny little bit, the drop from 60 to 30 ( 1 to 2 vertical retraces ) fps is extremely noticable.
-
I have also concluded that the networking architecture for Quake 2 was just not the right thing. The interpolating 10 hz server made a lot of animation easier, which fit with the single player focus, but it just wasn't a good thing for internet play.
Quake 3 will have an all new entity communication mechanism that should be solidly better than any previous system. I have some new ideas that go well beyond the previous work that I did on QuakeWorld.
Its tempting to try to roll the new changes back into Quake 2, but a lot of them are pretty fundamental, and I'm sure we would bust a lot of important single player stuff while gutting the network code.
(Yes, we made some direction changes in Quake 3 since the original announcement when it was to be based on the Quake 2 game and networking with just a new graphics engine)
-----------------------------------------
John Carmack's .plan for Jun 16, 1998
-----------------------------------------
My last two .plan updates have described efforts that were not in our original plan for quake 3, which was "quake 2 game and network technology with a new graphics engine".
We changed our minds.
The new product is going to be called "Quake Arena", and will consist exclusively of deathmatch style gaming (including CTF and other derivatives). The single player game will just be a progression through a ranking ladder against bot AIs. We think that can still be made an enjoyable game, but it is definately a gamble.
In the past, we have always been designing two games at once, the single player game and the multi player game, and they often had conflicting goals. For instance, the client-server communications channel discouraged massive quantities of moving entities that would have been interesting in single player, while the maps and weapons designed for single player were not ideal for multiplayer. The largest conflict was just raw development time. Time spent on monsters is time not spent on player movement. Time spent on unit goals is time not spent on game rules.
There are many wonderful gaming experiences in single player FPS, but we are choosing to leave them behind to give us a purity of focus that will let us make significant advances in the multiplayer experience.
The emphasis will be on making every aspect as robust and high quality as possible, rather than trying to add every conceivable option anyone could want. We will not be trying to take the place of every mod ever produced, but we hope to satisfy a large part of the network gaming audience with the out of box experience.
There is a definite effect on graphics technology decisions. Much of the positive feedback in a single player FPS is the presentation of rich visual scenes, which are often at the expense of framerate. A multiplayer level still needs to make a good first impression, but after you have seen it a hundred times, the speed of the game is more important. This means that there are many aggressive graphics technologies that I will not pursue because they are not apropriate to the type of game we are creating.
The graphics engine will still be OpenGL only, with significant new features not seen anywhere before, but it will also have fallback modes to render at roughly Quake-2 quality and speed.
-----------------------------------------
John Carmack's .plan for Jul 04, 1998
-----------------------------------------
Here is the real story on the movement physics changes.
Zoid changed the movement code in a way that he felt improved gameplay in the 3.15 release.
We don't directly supervise most of the work Zoid does. One of the main reasons we work with him is that I respect his judgment, and I feel that his work benefits the community quite a bit with almost no effort on my part. If I had to code review every change he made, it wouldn't be worth the effort.
Zoid has "ownership" of the Quake, Glquake, and QuakeWorld codebases. We don't intend to do any more modifications at Id on those sources, so he has pretty free reign within his discretion.
We passed over the Quake 2 codebase to him for the addition of new features like auto download, but it might have been a bit premature, because official mission packs were still in development, and unlike glquake and quakeworld, Q2 is a product that must remain official and supported, so the scope of his freedoms should have been spelled out a little more clearly.
The air movement code wasn't a good thing to change in Quake 2, because the codebase still had to support all the commercial single player levels, and subtle physics changes can have lots of unintended effects.
QuakeWorld didn't support single player maps, so it was a fine place to experiment with physics changes.
QuakeArena is starting with fresh new data, so it is also a good place to experiment with physics changes.
Quake 2 cannot be allowed to evolve in a way that detracts from the commercial single player levels.
The old style movement should not be referred to as "real world physics". None of the quake physics are remotely close to real world physics, so I don't think one way is significantly more "real" than the other. In Q2, you accelerate from 0 to 27 mph in 1/30 of a second, which just as unrealistic as being able to accelerate in midair..
-----------------------------------------
John Carmack's .plan for Jul 05, 1998
-----------------------------------------
I am not opposed to adding a flag to control the movement styles. I was rather expecting it to be made optional in 3.17, but I haven't been directly involved in the last few releases.
The way this played out in public is a bit unfortunate. Everyone at Id is busy full time with the new product, so we just weren't paying enough attention to the Quake 2 modifications. Some people managed to read into my last update that we were blaming Zoid for things. Uh, no. I think he was acting within his charter (catering to the community) very well, it just interfered with an aspect of the game that shouldn't have been modified. We just never made it explicitly clear that it shouldn't have been modified.
It is a bit amusing how after the QuakeArena anouncement, I got flamed by lots of people for abandoning single player play (even though we aren't, really) but after I say that Quake 2 can't forget that it is a single player game, I get flamed by a different set of people who think it is stupid to care about single player anymore when all "everyone" plays is multiplayer. The joy of having a wide audience that knows your email address.
-----------------------------------------
John Carmack's .plan for Jul 16, 1998
-----------------------------------------
I have spent the last two days working with Apple's Rhapsody DR2, and I like it a lot.
I was dissapointed with the original DR1 release. It was very slow and seemed to have added the worst elements of the mac experience (who the hell came up with that windowshade minimizing?) while taking away some of the strengths of NEXTSTEP.
Things are a whole lot better in the latest release. General speed is up, memory consumption is down, and the UI feels consistant and productive.
Its still not as fast as windows, and probably never will be, but I think the tradeoffs are valid.
There are so many things that are just fundamentally better in the rhapsody design than in windows: frameworks, the yellow box apis, fat binaries, buffered windows, strong multi user support, strong system / local seperation, netinfo, etc.
Right now, I think WindowsNT is the best place to do graphics development work, but if the 3D acceleration issue was properly addressed on rhapsody, I think that I could be happy using it as my primary development platform.
I ported the current Quake codebase to rhapsody to test out conix's beta OpenGL. The game isn't really playable with the software emulated OpenGL, but it functions properly, and it makes a fine dedicated server.
We are going to try to stay on top of the portability a little better for QA. Quake 2 slid a bit because we did the development on NT instead of NEXTSTEP, and that made the irix port a lot more of a hassle than the original glquake port.
I plan on using the rhapsody system as a dedicated server during development, and Brian will be using an Alpha-NT system for a lot of testing, which should give us pretty good coverage of the portability issues.
I'm supposed to go out and have a bunch of meetings at apple next month to cover games, graphics, and hardware. Various parts of apple have scheduled meetings with me on three seperate occasions over the past couple years, but they have always been canceled for one reason or another (they laid off the people I was going to meet with once..).
I have said some negative things about MacOs before, but my knowledge of the mac is five years old. There was certainly the possibility that things had improved since then, so I spent some time browsing mac documentation recently. I was pretty amused. A stack sniffer. Patching trap vectors. Cooperative multitasking. Application memory partitions. Heh.
I'm scared of MacOS X. As far as I can tell, The basic plan is to take rhapsody and bolt all the MacOS APIs into the kernel. I understand that that may well be a sensible biz direction, but I fear it.
In other operating system news, Be has glquake running hardware accelerated on their upcoming OpenGL driver architecture. I gave them access to the glquake and quake 2 codebases for development purposes, and I expect we will work out an agreement for distribution of the ports.
Any X server vendors working on hardware accelerated OpenGL should get in touch with Zoid about interfacing and tuning with the Id OpenGL games on linux.
-----------------------------------------
John Carmack's .plan for Jul 29, 1998
-----------------------------------------
My F50 took some twin turbo vitamins.
Rear wheel numbers: 602 HP @ 8200 rpm 418 ft-lb @ 7200 rpm
This is very low boost, but I got the 50% power increase I was looking for, and hopefully it won't be making any contributions to my piston graveyard.
There will be an article in Turbo magazine about it, and several other car magazines want to test it out. They usually start out with "He did WHAT to an F50???" :)
Brian is getting a nitrous kit installed in his viper, and Cash just got his suspension beefed up, so we will be off to the dragstrip again next month to sort everything out again.
-----------------------------------------
John Carmack's .plan for Aug 17, 1998
-----------------------------------------
I added support for HDTV style wide screen displays in QuakeArena, so 24" and 28" monitors can now cover the entire screen with game graphics.
On a normal 4:3 aspect ratio screen, a 90 degree horizontal field of view gives a 75 degree vertical field of view. If you keep the vertical fov constant and run on a wide screen, you get a 106 degree horizontal fov.
Because we specify fov with the horizontal measurement, you need to change fov when going into or out of a wide screen mode. I am considering changing fov to be the vertical measurement, but it would probably cause a lot of confusion if "fov 90" becomes a big fisheye.
Many video card drivers are supporting the ultra high res settings like 1920 * 1080, but hopefully they will also add support for lower settings that can be good for games, like 856 * 480.
I spent a day out at apple last week going over technical issues.
I'm feeling a lot better about MacOS X. Almost everything I like about rhapsody will be there, plus some solid additions.
I presented the OpenGL case directly to Steve Jobs as strongly as possible.
If Apple embraces OpenGL, I will be strongly behind them. I like OpenGL more than I dislike MacOS. :)
-
Last friday I got a phone call: "want to make some exhibition runs at the import / domestic drag wars this sunday?". It wasn't particularly good timing, because the TR had a slipping clutch and the F50 still hasn't gotten its computer mapping sorted out, but we got everything functional in time.
The tech inspector said that my cars weren't allowed to run in the 11s at the event because they didn't have roll cages, so I was supposed to go easy.
The TR wasn't running its best, only doing low 130 mph runs. The F50 was making its first sorting out passes at the event, but it was doing ok. My last pass was an 11.8(oops) @ 128, but we still have a ways to go to get the best times out of it.
I'm getting some racing tires on the F50 before I go back. It sucked watching a tiny honda race car jump ahead of me off the line. :)
I think ESPN took some footage at the event.
-----------------------------------------
John Carmack's .plan for Sep 08, 1998
-----------------------------------------
I just got a production TNT board installed in my Dolch today.
The riva-128 was a troublesome part. It scored well on benchmarks, but it had some pretty broken aspects to it, and I never recommended it (you are better off with an intel I740).
There aren't any troublesome aspects to TNT. Its just great. Good work, Nvidia.
In terms of raw speed, a 16 bit color multitexture app (like quake / quake 2) should still run a bit faster on a voodoo2, and an SLI voodoo2 should be faster for all 16 bit color rendering, but TNT has a lot of other things going for it:
32 bit color and 24 bit z buffers. They cost speed, but it is usually a better quality tradeoff to go one resolution lower but with twice the color depth.
More flexible multitexture combine modes. Voodoo can use its multitexture for diffuse lightmaps, but not for the specular lightmaps we offer in QuakeArena. If you want shiny surfaces, voodoo winds up leaving half of its texturing power unused (you can still run with diffuse lightmaps for max speed).
Stencil buffers. There aren't any apps that use it yet, but stencil allows you to do a lot of neat tricks.
More texture memory. Even more than it seems (16 vs 8 or 12), because all of the TNT's memory can be used without restrictions. Texture swapping is the voodoo's biggest problem.
3D in desktop applications. There is enough memory that you don't have to worry about window and desktop size limits, even at 1280*1024 true color resolution.
Better OpenGL ICD. 3dfx will probably do something about that, though.
This is the shape of 3D boards to come. Professional graphics level rendering quality with great performance at a consumer price.
We will be releasing preliminary QuakeArena benchmarks on all the new boards in a few weeks. Quake 2 is still a very good benchmark for moderate polygon counts, so our test scenes for QA involve very high polygon counts, which stresses driver quality a lot more. There are a few surprises in the current timings..
-
A few of us took a couple days off in vegas this weekend. After about ten hours at the tables over friday and saturday, I got a tap on the shoulder..
Three men in dark suits introduced themselves and explained that I was welcome to play any other game in the casino, but I am not allowed to play blackjack anymore.
Ah well, I guess my blackjack days are over. I was actually down a bit for the day when they booted me, but I made +$32k over five trips to vegas in the past two years or so.
I knew I would get kicked out sooner or later, because I don't play "safely". I sit at the same table for several hours, and I range my bets around 10 to 1.
-----------------------------------------
John Carmack's .plan for Sep 10, 1998
-----------------------------------------
I recently set out to start implementing the dual-processor acceleration for QA, which I have been planning for a while. The idea is to have one processor doing all the game processing, database traversal, and lighting, while the other processor does absolutely nothing but issue OpenGL calls.
This effectively treats the second processor as a dedicated geometry accelerator for the 3D card. This can only improve performance if the card isn't the bottleneck, but voodoo2 and TNT cards aren't hitting their limits at 640*480 on even very fast processors right now.
For single player games where there is a lot of cpu time spent running the server, there could conceivably be up to an 80% speed improvement, but for network games and timedemos a more realistic goal is a 40% or so speed increase. I will be very satisfied if I can makes a dual pentium-pro 200 system perform like a pII-300.
I started on the specialized code in the renderer, but it struck me that it might be possible to implement SMP acceleration with a generic OpenGL driver, which would allow Quake2 / sin / halflife to take advantage of it well before QuakeArena ships.
It took a day of hacking to get the basic framework set up: an smpgl.dll that spawns another thread that loads the original oepngl32.dll or 3dfxgl.dll, and watches a work que for all the functions to call.
I get it basically working, then start doing some timings. Its 20% slower than the single processor version.
I go in and optimize all the queing and working functions, tune the communications facilities, check for SMP cache collisions, etc.
After a day of optimizing, I finally squeak out some performance gains on my tests, but they aren't very impressive: 3% to 15% on one test scene, but still slower on the another one.
This was fairly depressing. I had always been able to get pretty much linear speedups out of the multithreaded utilities I wrote, even up to sixteen processors. The difference is that the utilities just split up the work ahead of time, then don't talk to each other until they are done, while here the two threads work in a high bandwidth producer / consumer relationship.
I finally got around to timing the actual communication overhead, and I was appalled: it was taking 12 msec to fill the que, and 17 msec to read it out on a single frame, even with nothing else going on. I'm surprised things got faster at all with that much overhead.
The test scene I was using created about 1.5 megs of data to relay all the function calls and vertex data for a frame. That data had to go to main memory from one processor, then back out of main memory to the other. Admitedly, it is a bitch of a scene, but that is where you want the acceleration..
The write times could be made over twice as fast if I could turn on the PII's write combining feature on a range of memory, but the reads (which were the gating factor) can't really be helped much.
Streaming large amounts of data to and from main memory can be really grim. The next write may force a cache writeback to make room for it, then the read from memory to fill the cacheline (even if you are going to write over the entire thing), then eventually the writeback from the cache to main memory where you wanted it in the first place. You also tend to eat one more read when your program wants to use the original data that got evicted at the start.
What is really needed for this type of interface is a streaming read cache protocol that performs similarly to the write combining: three dedicated cachelines that let you read or write from a range without evicting other things from the cache, and automatically prefetching the next cacheline as you read.
Intel's write combining modes work great, but they can't be set directly from user mode. All drivers that fill DMA buffers (like OpenGL ICDs..) should definately be using them, though.
Prefetch instructions can help with the stalls, but they still don't prevent all the wasted cache evictions.
It might be possible to avoid main memory alltogether by arranging things so that the sending processor ping-pongs between buffers that fit in L2, but I'm not sure if a cache coherent read on PIIs just goes from one L2 to the other, or if it becomes a forced memory transaction (or worse, two memory transactions). It would also limit the maximum amount of overlap in some situations. You would also get cache invalidation bus traffic.
I could probably trim 30% of my data by going to a byte level encoding of all the function calls, instead of the explicit function pointer / parameter count / all-parms-are-32-bits that I have now, but half of the data is just raw vertex data, which isn't going to shrink unless I did evil things like quantize floats to shorts.
Too much effort for what looks like a reletively minor speedup. I'm giving up on this aproach, and going back to explicit threading in the renderer so I can make most of the communicated data implicit.
Oh well. It was amusing work, and I learned a few things along the way.
-----------------------------------------
John Carmack's .plan for Oct 14, 1998
-----------------------------------------
It has been difficult to write .plan updates lately. Every time I start writing something, I realize that I'm not going to be able to cover it satisfactorily in the time I can spend on it. I have found that terse little comments either get misinterpreted, or I get deluged by email from people wanting me to expand upon it.
I wanted to do a .plan about my evolving thoughts on code quality and lessons learned through quake and quake 2, but in the interest of actually completing an update, I decided to focus on one change that was intended to just clean things up, but had a surprising number of positive side effects.
Since DOOM, our games have been defined with portability in mind. Porting to a new platform involves having a way to display output, and having the platform tell you about the various relevant inputs. There are four principle inputs to a game: keystrokes, mouse moves, network packets, and time. (If you don't consider time an input value, think about it until you do - it is an important concept)
These inputs were taken in separate places, as seemed logical at the time. A function named Sys_SendKeyEvents() was called once a frame that would rummage through whatever it needed to on a system level, and call back into game functions like Key_Event( key, down ) and IN_MouseMoved( dx, dy ). The network system dropped into system specific code to check for the arrival of packets. Calls to Sys_Milliseconds() were littered all over the code for various reasons.
I felt that I had slipped a bit on the portability front with Q2 because I had been developing natively on windows NT instead of cross developing from NEXTSTEP, so I was reevaluating all of the system interfaces for Q3.
I settled on combining all forms of input into a single system event queue, similar to the windows message queue. My original intention was to just rigorously define where certain functions were called and cut down the number of required system entry points, but it turned out to have much stronger benefits.
With all events coming through one point (The return values from system calls, including the filesystem contents, are "hidden" inputs that I make no attempt at capturing, ), it was easy to set up a journalling system that recorded everything the game received. This is very different than demo recording, which just simulates a network level connection and lets time move at its own rate. Realtime applications have a number of unique development difficulties because of the interaction of time with inputs and outputs.
Transient flaw debugging. If a bug can be reproduced, it can be fixed. The nasty bugs are the ones that only happen every once in a while after playing randomly, like occasionally getting stuck on a corner. Often when you break in and investigate it, you find that something important happened the frame before the event, and you have no way of backing up. Even worse are realtime smoothness issues - was that jerk of his arm a bad animation frame, a network interpolation error, or my imagination?
Accurate profiling. Using an intrusive profiler on Q2 doesn't give accurate results because of the realtime nature of the simulation. If the program is running half as fast as normal due to the instrumentation, it has to do twice as much server simulation as it would if it wasn't instrumented, which also goes slower, which compounds the problem. Aggressive instrumentation can slow it down to the point of being completely unplayable.
Realistic bounds checker runs. Bounds checker is a great tool, but you just can't interact with a game built for final checking, its just waaaaay too slow. You can let a demo loop play back overnight, but that doesn't exercise any of the server or networking code.
The key point: Journaling of time along with other inputs turns a realtime application into a batch process, with all the attendant benefits for quality control and debugging. These problems, and many more, just go away. With a full input trace, you can accurately restart the session and play back to any point (conditional breakpoint on a frame number), or let a session play back at an arbitrarily degraded speed, but cover exactly the same code paths..
I'm sure lots of people realize that immediately, but it only truly sunk in for me recently. In thinking back over the years, I can see myself feeling around the problem, implementing partial journaling of network packets, and included the "fixedtime" cvar to eliminate most timing reproducibility issues, but I never hit on the proper global solution. I had always associated journaling with turning an interactive application into a batch application, but I never considered the small modification necessary to make it applicable to a realtime application.
In fact, I was probably blinded to the obvious because of one of my very first successes: one of the important technical achievements of Commander Keen 1 was that, unlike most games of the day, it adapted its play rate based on the frame speed (remember all those old games that got unplayable when you got a faster computer?). I had just resigned myself to the non-deterministic timing of frames that resulted from adaptive simulation rates, and that probably influenced my perspective on it all the way until this project.
Its nice to see a problem clearly in its entirety for the first time, and know exactly how to address it.
-----------------------------------------
John Carmack's .plan for Nov 03, 1998
-----------------------------------------
This was the most significant thing I talked about at The Frag, so here it is for everyone else.
The way the QA game architecture has been developed so far has been as two seperate binary dll's: one for the server side game logic, and one for the client side presentation logic.
While it was easiest to begin development like that, there are two crucial problems with shipping the game that way: security and portability.
It's one thing to ask the people who run dedicated servers to make informed decisions about the safety of a given mod, but its a completely different matter to auto-download a binary image to a first time user connecting to a server they found.
The quake 2 server crashing attacks have certainly proven that there are hackers that enjoy attacking games, and shipping around binary code would be a very tempting opening for them to do some very nasty things.
With quake and Quake 2, all game modifications were strictly server side, so any port of the game could connect to any server without problems. With Quake 2's binary server dll's not all ports could necessarily run a server, but they could all play.
With significant chunks of code now running on the client side, if we stuck with binary dll's then the less popular systems would find that they could not connect to new servers because the mod code hadn't been ported. I considered having things set up in such a way that client game dll's could be sort of forwards-compatable, where they could always connect and play, but new commands and entity types just might now show up. We could also GPL the game code to force mod authors to release source with the binaries, but that would still be inconvenient to deal with all the porting.
Related both issues is client side cheating. Certain cheats are easy to do if you can hack the code, so the server will need to verify which code the client is running. With multiple ported versions, it wouldn't be possible to do any binary verification.
If we were willing to wed ourselves completely to the windows platform, we might have pushed ahead with some attempt at binary verification of dlls, but I ruled that option out. I want QuakeArena running on every platform that has hardware accelerated OpenGL and an internet connection.
The only real solution to these problems is to use an interpreted language like Quake 1 did. I have reached the conclusion that the benefits of a standard language outweigh the benefits of a custom language for our purposes. I would not go back and extend QC, because that stretches the effort from simply system and interpreter design to include language design, and there is already plenty to do.
I had been working under the assumption that Java was the right way to go, but recently I reached a better conclusion.
The programming language for QuakeArena mods is interpreted ANSI C. (well, I am dropping the double data type, but otherwise it should be pretty conformant)
The game will have an interpreter for a virtual RISC-like CPU. This should have a minor speed benefit over a byte-coded, stack based java interpreter. Loads and stores are confined to a preset block of memory, and access to all external system facilities is done with system traps to the main game code, so it is completely secure.
The tools necessary for building mods will all be freely available: a modified version of LCC and a new program called q3asm. LCC is a wonderful project - a cross platform, cross compiling ANSI C compiler done in under 20K lines of code. Anyone interested in compilers should pick up a copy of "A retargetable C compiler: design and implementation" by Fraser and Hanson.
You can't link against any libraries, so every function must be resolved. Things like strcmp, memcpy, rand, etc. must all be implemented directly. I have code for all the ones I use, but some people may have to modify their coding styles or provide implementations for other functions.
It is a fair amount of work to restructure all the interfaces to not share pointers between the system and the games, but it is a whole lot easier than porting everything to a new language. The client game code is about 10k lines, and the server game code is about 20k lines.
The drawback is performance. It will probably perform somewhat like QC. Most of the heavy lifting is still done in the builtin functions for path tracing and world sampling, but you could still hurt yourself by looping over tons of objects every frame. Yes, this does mean more load on servers, but I am making some improvements in other parts that I hope will balance things to about the way Q2 was on previous generation hardware.
There is also the amusing avenue of writing hand tuned virtual assembly assembly language for critical functions..
I think this is The Right Thing.
-----------------------------------------
John Carmack's .plan for Nov 04, 1998
-----------------------------------------
More extensive comments on the interpreted-C decision later, but a quick note: the plan is to still allow binary dll loading so debuggers can be used, but it should be interchangable with the interpreted code. Client modules can only be debugged if the server is set to allow cheating, but it would be possible to just use the binary interface for server modules if you wanted to sacrifice portability. Most mods will be able to be implemented with just the interpreter, but some mods that want to do extensive file access or out of band network communications could still be implemented just as they are in Q2. I will not endorse any use of binary client modules, though.
-----------------------------------------
John Carmack's .plan for Dec 29, 1998
-----------------------------------------
I am considering taking a shortcut with my virtual machine implementation that would make the integration a bit easier, but I'm not sure that it doesn't compromise the integrity of the base system.
I am considering allowing the interpreted code to live in the global address space, instead of a private 0 based address space of its own. Store instructions from the VM would be confined to the interpreter's address space, but loads could access any structures.
On the positive side:
This would allow full speed (well, full interpreted speed) access to variables shared between the main code and the interpreted modules. This allows system calls to return pointers, instead of filling in allocated space in the interpreter's address space.
For most things, this is just a convenience that will cut some development time. Most of the shared accesses could be recast as "get" system calls, and it is certainly arguable that that would be a more robust programming style.
The most prevelent change this would prevent is all the cvar_t uses. Things could stay in the same style as Q2, where cvar accesses are free and transparantly updated. If the interpreter lives only in its own address space, then cvar access would have to be like Q1, where looking up a variable is a potentially time consuming operation, and you wind up adding lots of little cvar caches that are updated every from or restart.
On the negative side:
A client game module with a bug could cause a bus error, which would not be possible with a pure local address space interpreter.
I can't think of any exploitable security problems that read only access to the entire address space opens, but if anyone thinks of something, let me know.
-----------------------------------------
John Carmack's .plan for Dec 30, 1998
-----------------------------------------
I got several vague comments about being able to read "stuff" from shared memory, but no concrete examples of security problems.
However, Gregory Maxwell pointed out that it wouldn't work cross platform with 64 bit pointer environments like linux alpha. That is a killer, so I will be forced to do everything the hard way. Its probably for the best, from a design standpoint anyway, but it will take a little more effort.
-----------------------------------------
John Carmack's .plan for Jan 10, 1999
-----------------------------------------
Ok, many of you have probably heard that I spoke at the macworld keynote on tuesday. Some information is probably going to get distorted in the spinning and retelling, so here is an info dump straight from me:
Q3test, and later the full commercial Quake3: Arena, will be simultaniously released on windows, mac, and linux platforms.
I think Apple is doing a lot of things right. A lot of what they are doing now is catch-up to wintel, but if they can keep it up for the next year, they may start making a really significant impact.
I still can't give the mac an enthusiastic recommendation for sophisticated users right now because of the operating system issues, but they are working towards correcting that with MacOS X.
The scoop on the new G3 mac hardware:
Basically, its a great system, but Apple has oversold its performance reletive to intel systems. In terms of timedemo scores, the new G3 systems should be near the head of the pack, but there will be intel systems outperforming them to some degree. The mac has not instantly become a "better" platform for games than wintel, it has just made a giant leap from the back of the pack to near the front.
I wish Apple would stop quoting "Bytemarks". I need to actually look at the contents of that benchmark and see how it can be so misleading. It is pretty funny listening to mac evangelist types try to say that an iMac is faster than a pentium II-400. Nope. Not even close.
From all of my tests and experiments, the new mac systems are basically as fast as the latest pentium II systems for general cpu and memory performance. This is plenty good, but it doesn't make the intel processors look like slugs.
Sure, an in-cache, single precision, multiply-accumulate loop could run twice as fast as a pentium II of the same clock rate, but conversly, a double precision add loop would run twice as fast on the pentium II.
Spec95 is a set of valid benchmarks in my opinion, and I doubt the PPC systems significantly (if at all) outperform the intel systems.
The IO system gets mixed marks. The 66 mhz video slot is a good step up from 33 mhz pci in previous products, but that's still half the bandwidth of AGP 2X, and it can't texture from main memory. This will have a small effect on 3D gaming, but not enough to push it out of its class.
The 64 bit pci slots are a good thing for network and storage cards, but the memory controller doesn't come close to getting peak utilization out of it. Better than normal pci, though.
The video card is almost exactly what you will be able to get on the pc side: a 16 mb rage-128. Running on a 66mhz pci bus, it's theoretical peak performance will be midway between the pci and agp models on pc systems for command traffic limited scenes. Note that current games are not actually command traffic limited, so the effect will be significantly smaller. The fill rates will be identical.
The early systems are running the card at 75 mhz, which does put it at a slight disadvantage to the TNT, but faster versions are expected later. As far as I can tell, the rage-128 is as perfect as the TNT feature-wise. The 32 mb option is a feature ATI can hold over TNT.
Firewire is cool.
Its a simple thing, but the aspect of the new G3 systems that struck me the most was the new case design. Not the flashy plastic exterior, but the functional structure of it. The side of the system just pops open, even with the power on, and lays the motherboard and cards down flat while the disks and power supply stay in the encloser. It really is a great design, and the benefits were driven home yesterday when I had to scavenge some ram out of old wintel systems yesterday - most case designs suck really bad.
-
I could gripe a bit about the story of our (lack of) involvement with Apple over the last four years or so, but I'm calling that water under the bridge now.
After all the past fiascos, I had been basically planning on ignoring Apple until MacOS X (rhapsody) shipped, which would then turn it into a platform that I was actually interested in.
Recently, Apple made a strategic corporate decision that games were a fundamental part of a consumer oriented product line (duh). To help that out, Apple began an evaluation of what it needed to do to help game developers.
My first thought was "throw out MacOS", but they are already in the process of doing that, and its just not going to be completed overnight.
Apple has decent APIs for 2D graphics, input, sound, and networking, but they didn't have a solid 3D graphics strategy.
Rave was sort of ok. Underspecified and with no growth path, but sort of ok. Pursuing a proprietary api that wasn't competetive with other offerings would have been a Very Bad Idea. They could have tried to make it better, or even invent a brand new api, but Apple doesn't have much credebility in 3D programming.
For a while, it was looking like Apple might do something stupid, like license DirectX from microsoft and be put into a guaranteed trailing edge position behind wintel.
OpenGL was an obvious direction, but there were significant issues with the licensing and implementation that would need to be resolved.
I spent a day at apple with the various engineering teams and executives, laying out all the issues.
The first meeting didn't seem like it went all that well, and there wasn't a clear direction visible for the next two months. Finally, I got the all clear signal that OpenGL was a go and that apple would be getting both the sgi codebase and the conix codebease and team (the best possible arrangement).
So, I got a mac and started developing on it. My first weekend of effort had QuakeArena limping along while held together with duct tape, but weekend number two had it properly playable, and weekend number three had it brought up to full feature compatability. I still need to do some platform specific things with odd configurations like multi monitor and addon controlers, but basically now its just a matter of compiling on the mac to bring it up to date.
This was important to me, because I felt that Quake2 had slipped a bit in portability because it had been natively developed on windows. I like the discipline of simultanious portable development.
After 150 hours or so of concentrated mac development, I learned a lot about the platform.
CodeWarrior is pretty good. I was comfortable devloping there almost immediately. I would definately say VC++ 6.0 is a more powerful overall tool, but CW has some nice little aspects that I like. I am definately looking forward to CW for linux. Unix purists may be aghast, but I have allways liked gui dev environments more than a bunch of xterms running vi and gdb.
The hardware (even the previous generation stuff) is pretty good.
The OpenGL performance is pretty good. There is a lot of work underway to bring the OpenGL performance to the head of the pack, but the existing stuff works fine for development.
The low level operating systems SUCKS SO BAD it is hard to believe.
The first order problem is lack of memory management / protection.
It took me a while to figure out that the zen of mac development is "be at peace while rebooting". I rebooted my mac system more times the first weekend than I have rebooted all the WinNT systems I have ever owned. True, it has gotten better now that I know my way around a bit more, and the codebase is fully stable, but there is just no excuse for an operating system in this day and age to act like it doesn't have access to memory protection.
The first thing that bit me was the static memory allocation for the program. Modern operating systems just figure out how much memory you need, but because the mac was originally designed for systems without memory management, significant things have to be laid out ahead of time.
Porting a win32 game to the mac will probably involve more work dealing with memory than any other aspect. Graphics, sound, and networking have reasonable analogues, but you just can't rely on being able to malloc() whatever you want on the mac.
Sure, game developers can manage their own memory, but an operating system that has proper virtual memory will let you develop a lot faster.
The lack of memory protection is the worst aspect of mac development. You can just merrily write all over other programs, the development environment, and the operating system from any application.
I remember that. From dos 3.3 in 1990.
Guard pages will help catch simple overruns, but it won't do anything for all sorts of other problems.
The second order problem is lack of preemptive multitasking.
The general responsiveness while working with multiple apps is significantly worse than windows, and you often run into completely modal dialogs that don't let you do anything else at all.
A third order problem is that a lot of the interfaces are fairly clunky.
There are still many aspects of the mac that clearly show design decisions based on a 128k 68000 based machine. Wintel has grown a lot more than the mac platform did. It may have been because the intel architecture didn't evolve gracefully and that forced the software to reevaluate itself more fully, or it may just be that microsoft pushed harder.
Carbon sanitizes the worst of the crap, but it doesn't turn it into anything particularly good looking.
MacOS X nails all these problems, but thats still a ways away.
I did figure one thing out - I was always a little curious why the early BeOS advocates were so enthusiastic. Coming from a NEXTSTEP background, BeOS looked to me like a fairly interesting little system, but nothing special. To a mac developer, it must have looked like the promised land..
-----------------------------------------
John Carmack's .plan for Jan 29, 1999
-----------------------------------------
The issue of the 8192 unit map dimension limit had been nagging at me for a long time now. The reason for the limit is that coordinates are communicated over the network in 16 bit shorts divided as a sign bit, twelve unit bits, and three fractional bits. There was also another side to the representation problem, but it was rarely visible: if you timescale down, you can actually tell the fractional bit granularity in position and velocity.
The rest of the system (rendering and gameplay) has never had any issues with larger maps, even in quake 1. There are some single precision floating point issues that begin to creep in if things get really huge, but maps should be able to cover many times the current limit without any other changes.
A while ago I had changed Q3 so that the number of fractional bits was a compile time option, which allowed you to trade off fine grain precision for larger size. I was considering automatically optimizing this for each level based on its size, but it still didn't feel like a great solution.
Another aspect of the problem that wasn't visible to the public was that the the fractional quantization of position could cause the position to actually be inside a nearby solid when used for client side prediction. The code had to check for this and try to correct the situation by jittering the position in each of the possible directions it might have been truncated from. This is a potential issue whenever there is any loss of precision whatsoever in the server to client communication.
The obvious solution is to just send the full floating point value for positions, but I resisted that because the majority of our network traffic is positional updates, and I didn't want to bloat it. There have been other bandwidth savings in Q3, and LANs and LPB connections are also relevent, so I was constantly evaluating the tradeoff.
Dealing with four or five players in view isn't a real problem. The big bandwidth issues arrive when multiple players start unloading with rapid fire weapons. (as an aside, I tried making 5hz fire weapons for Q3 to save bandwidth, but no matter how much damage they did, 5hz fire rates just seemed to feel slow and weak...)
I finally moved to a bit-level stream encoding to save some more bandwidth and give me some more representational flexibility, and this got me thinking about the characteristics of the data that bother us.
In general, the floating point coordinates have significant bits all through the mantissa. Any movement along an angle will more or less randomize the low order bits.
My small little insight was that because missiles are evaluated parameterically instead of itteretively in Q3, a one-time snapping of the coordinates can be performed at their generation time, giving them fixed values with less significant bits for their lifetime without any effort outside their spawning function. It also works for doors and plats, which are also parametrically represented now. Most events will also have integral coordinates.
The float encoder can check for an integral value in a certain range and send that as a smaller number of bits, say 13 or so. If the value isn't integral, it will be transmitted as a full 32 bit float.
The other thing I am investigating is sub-byte delta encoding of floating point values. Even with arbitrary precision movement deltas, the sign and exponent bits change with very low frequency except when you are very near the origin. At the minimum, I should be able to cut the standard player coordinate delta reps to three bytes from four.
So, the bottom line is that the bandwidth won't move much (it might even go down if I cut the integral bits below 15), the maps become unbounded in size to the point of single precision roundoff, and the client doesn't have to care about position jittering (which was visible in Q3 code that will be released).
-----------------------------------------
John Carmack's .plan for Mar 03, 1999
-----------------------------------------
On the issue of railgun firing rates -- we played with it for a while at the slower speed, but it has been put back to exactly Q2's rate of fire.
I do agree with Thresh that the way we had it initially (faster than Q2, but with the same damage) made it an overpowered weapon in the hands of highly skilled players, which is exactly what we should try to avoid.
An ideal game should give scores as close to directly proportional to the players reletive skills as possible. The better player should win in almost all cases, but the game will be more entertaining if the inferior players are not completely dominated.
Quake 1 had really bad characteristics that way -- Thresh can play extremely talented players and often prevent them from scoring a single point. We wouldn't put up with a conventional sport that commonly game scores of 20 to 1 in championship matches, and I don't think we should encourage it in our games.
Eliminating health items is probably the clearest way to prevent blowout games, but that has never been popular. Still, we should try to avoid weapon decisions that allow the hyper-skilled to pull even farther away from the rest of the crowd. They will still win, no matter what the weapons are, just not by as wide a margin.
-----------------------------------------
John Carmack's .plan for Mar 17, 1999
-----------------------------------------
First impressions of the SGI visual workstation 320:
I placed an order for a loaded system ($11k) from their web site two months ago. It still hasn't arrived (bad impression), but SGI did bring a loaner system by for us to work with.
The system tower is better than standard pc fare, but I still think Apple's new G3 has the best designed computer case.
The wide aspect LCD panel is very nice. A while ago I had been using a dual monitor LCD setup on a previous intergraph, but the analog syncing LCD screens had some fringing problems, and the gamma range was fairly different from a CRT. The SGI display is perfectly crisp (digital interface), and has great color. The lighting is a little bit uneven top to bottom on this unit -- I am interested to see how the next unit looks for comparison.
Unfortunately, the card that they bundle with the LCD if you buy it separately is NOT a good 3D accelerator, so if you care about 3D and want this LCD screen, you need to buy an sgi visual workstation. Several of the next generation consumer cards are going to support digital flat panel outputs, so hopefully soon you will be able to run a TNT2 or something out to one of these.
The super memory system does not appear to have provided ANY benefit to the CPU. My memory benchmarking tests showed it running about the same as a standard intel design.
Our first graphics testing looked very grim -- Quake3 didn't draw the world at all. I spent a while trying to coax some output by disabling various things, but to no avail. We reported it to SGI, and they got us a fix the next day. Some bug with depthRange(). Even with the fix, 16 bit rendering doesn't seem to work. I expect they will address this.
Other than that, there haven't been any driver anomolies, and both the game and editor run flawlessly.
For single pass, top quality rendering (32 bit framebuffer, 32 bit depth buffer, 32 bit trilinear textures, high res screen), the SGI has a higher fill rate than any other card we have ever tested on a pc, but not by too wide of a margin.
If your application can take advantage of multitexture, a TNT or rage128 will deliver slightly greater fill performance. It is likely that the next speed bump of both chips will be just plain faster than the SGI on all fill modes.
A serious flaw is that the LCD display can't support ANY other resolutions except the native 1600*1024. The game chunks along at ten to fifteen fps at that resolution (but it looks cool!). They desperately need to support a pixel doubled 800*512 mode to make any kind of full screen work possible. I expect they will address this.
Vsync disable is implemented wrong. Disabling sync causes it to continue rendering, but the flip still doesn't happen until the next frame. This gives repeatable (and faster) benchmark numbers, but with a flashing screen that is unusable. The right way is to just cause the flip to happen on the next scan line, like several consumer cards do, or blit. It gives tearing artifacts, but it is still completely usable, and avoids temporal nyquist issues between 30 and 60 hz. I expect they will address this.
Total throughput for games is only fair, about like an intergraph. Any of the fast consumer cards will run a quake engine game faster than the sgi in its current form. I'm sure some effort will be made to improve this, but I doubt it will be a serious focus, unless some SGI engineers develop unhealthy quake addictions. :-)
The unified memory system allows nearly a gig of textures, and individual textures can by up to 4k by 4k. AGP texturing provides some of this benefit for consumer cards, but not to the same degree or level of performance.
The video stream support looks good, but I haven't tried using it yet.
Very high interpolater accuracy. All the consumer cards start to break up a bit with high magnification, weird aspects, or long edges. The professional cards (intergraph, glint, E&S, SGI) still do a better job.
SGI exports quite a few more useful OpenGL extensions than intergraph does, but multisample antialiasing (as claimed in their literature) doesn't seem to be one of them.
Overall, it looks pretty good, and I am probably going to move over to using the SGI workstation full time when my real system arrives.
I was very happy with the previous two generations of intergraph workstations, but this last batch (GT1) has been a bunch of lemons, and the wildcat graphics has been delayed too long. The current realizm-II systems just don't have enough fill rate for high end development.
For developers that don't have tons of money, the decision right now is an absolute no-brainer -- buy a TNT and stick it in a cheap system. Its a better "professional" 3D accelerator than you could buy at any price not too long ago.
-----------------------------------------
John Carmack's .plan for Apr 24, 1999
-----------------------------------------
We are finally closing in on the first release of Q3test.
As you have probably heard by now, the first release in going to be the mac version, probably followed by the linux version, and only then the windows version.
Some of you are busy getting all bent out of shape about this.
We want to get a lot of people playing with the test to find bugs and offer well thought out suggestions, but there are classes of bugs and suggestions that emerge in each order of magnitude of exposed testers.
If a given bug is going to show up when a thousand people have looked at it, but we had released it to a hundred thousand people, then we are going to have a lot of duplication to wade through.
The mac testers will find some obvious problems. We will fix them. The later releases will be better.
Even if we had the windows distribution ready to go right now, I would still seriously consider releasing one of the mac or linux versions first because it will make our tracking a lot easier.
The holdup on the windows side are the issues with updated driver distribution. The game itself doesn't have any holdups.
We could do a windows release for just, say, voodoo2 owners and get some of the benefit of a controlled release, but it wouldn't really work out, because everyone would figure out that it can be made to (almost) work on lots of other cards by tweaking some values. That type of feedback would not be useful, because we KNOW that there are problems with most of the current drivers. We have been working with all of the vendors for the past year to get them all straightened out.
Glsetup is going to be slick -- just run it and it will Do The Right Thing for your video configuration.
We hope it will be done soon, but there are factors out of our direct control involved.
Don't be spiteful. This is just the beginning of the testing and release process.
One conspiracy theory suggests that Apple is somehow getting us to do this.
What we have "gotten" from Apple is a few development machines. No cash payoff. No bundling deal. No marketing contract.
I am looking at this long term. I want to see OS X become a top notch platform for graphics development. I think highly of the NEXTSTEP heritage and I might move my development from NT if it turns out well. There is a lot of groundwork that needs to be laid with apple for this to happen, and my working on the mac right now is part of that. Plus a lot of complaining to various apple engineers and executives. :-)
To be clear:
At this time, there is no mac that is as fast for gaming (or just about anything, actually) as a pentium III with a top of the line 3D card. Period. I have been misquoted by some mac evangelists as saying otherwise.
The new (blue and white) G3 systems are very good systems in many ways, and make a perfectly good gaming platform. However, a high end wintel machine just has more horsepower on both the CPU and the 3D card.
A 400 mhz G3 performs about the same as a 400 mhz PII if they aren't fill rate limited, where the faster cards on the PC will give about a 25% advantage. A 500 mhz PIII with an appropriate card in 30% faster than the best mac you can buy.
The multi colored iMacs, old G3 desktops, and powerbooks can play Quake3, but the RagePro 3D acceleration defines the absolute bottom end of our supported platforms. A serious gamer will not be satisfied with it.
Voodoo cards are not currently supported by the OpenGL driver, which is very unfortunate because many serious mac gamers own voodoo cards. I hope Apple and 3dfx will be able to correct this soon, but I certainly understand Apple's prioritization -- obviously, good support for the OEM video options is of primary importance.
The voodoo performance will still lag the windows platform by some amount, but some strides have been made in that area recently, so I expect good performance,
Gaming is not a reason to buy a mac, but Apple is taking steps so that it may not be a reason to avoid a mac if you have other reasons for wanting one.
MacOS still sucks.
-----------------------------------------
John Carmack's .plan for Apr 25, 1999
-----------------------------------------
Some people seem to think that I just make up these performance comparison numbers. I don't. I measure things, and I understand control conditions.
In this discussion, assume "wintel" is a 500 mhz PIII with either a agp rage128, or an agp TNT card, and "macos" is a 400 mhz G3 with the pci rage128.
At the highest level, you can make application class comparisons between platforms. For instance, CodeWarrior on the mac compiles faster than VC++ on wintel, but stuffit is way slower than winzip. This is useful data, but says more about the application design than the relative merits of the platforms. CW uses a single object file repository, for instance.
A better comparison is an identical app on both platforms.
Photoshop is often faster on macos than wintel. There is certainly a lot of common code, but individual filters are optimized for each platform. Some of these hand optimized operations are significantly faster on the mac.
Quake1 was the counterpoint to that. Quake1 had significant amounts of hand tuned asm code for intel, and the PPC version never got as much attention. The PPC version was noticeably slower (you would have to time at 640*480 to avoid unfairly penalizing the mac for lack of low res modes).
So, clearly, hand tuned asm code can make either platform pull ahead. It also shows that the two platforms are at least fairly close in performance. I never said macs were SLOW, just not quite as fast as the best intel systems.
Quake3 doesn't software rasterize, so there isn't any great place for lots of asm code (the great place is in the OpenGL driver). The code is essentially identical on all platforms.
Q3 is definitely faster on a wintel system than a macos system. When the wintel version is released, everyone will be independantly repeating that measurement.
Even this measurement isn't exactly an apples to apples comparison, because the OpenGL driver and 3D card are still a significant variance. The two can be broken out farther: Q3 can be run without 3D output to test just the identical compiled code. Wintel is still faster, although somewhat less so. The OpenGL + 3D card setup can be benchmarked separately on the axis of throughput and fill rate, which show the intel system being significantly faster. I can't break that apart into the two separate components, but I will guess that the OpenGL driver is probably as efficient as the wintel drivers and the performance delta is due to the system interface and the video card. The current mac rage128 cards run at 75 mhz, which is a third slower than the PC cards. AGP is also more than just a faster PCI, it can influence the structure of communication with the card.
It has been my observation in the past that most of my code tracks just about midway between specint and specfp for performance comparisons. There is a lot of floating point, but it is all short vectors, rather than the massive vectors of scientific computing. If we discount the graphics subsystem, Q3 follows this reasonably well. The intel system does slightly better than projected.
"Sucks" is a subjective description that can be dismissed as opinion. Note that I have NEVER said that the hardware sucks, or the user interface sucks, just that the mac OPERATING SYSTEM sucks.
"Faster", when qualified with testing conditions, is objective, and all the wishing in the world doesn't change it.
Objectivity and quantification are the paths to improvement.
I will be very happy if Apple can produce a desktop system that is faster than anything else you can get. I respect good engineering from any source. Altivec should be better than the PIII extensions (trinary ops -- yeah!). The upcoming system architectures look good. They have a shot at it, but they won't make it if they complacently think "oh, we are already faster than any pc system".
My twin turbo F50 can still be outrun at the dragstrip by much cheaper race cars. Many ferrari owners would not dare set foot at a drag strip, because they fear objective measurements that may not show their important possession in the best light. I would rather have the facts, so I can base future decisions on logical grounds.
-----------------------------------------
John Carmack's .plan for Apr 26, 1999
-----------------------------------------
Interpreting the lagometer (the graph in the lower right corner):
The upper graph (blue/yellow) slides one pixel for every rendered frame. Blue lines below the baseline mean that the frame is interpolating between two valid snapshots. Yellow lines above the baseline mean the frame is extrapolating beyond the latest valid time. The length of the line is proportional to the time.
The lower graph (green/yellow/red) slides one pixel for every received snapshot. By default, snapshots come 20 times a second, so if you are running >20 fps, the top graph will move faster, and vice versa. A red bar means the snapshot was dropped by the network. Green and yellow bars are properly received snapshots, with the height of the bar proportional to the ping. A yellow bar indicates that the previous snapshot was intentionally supressed to stay under the rate limit.
The upper graph indicates the consistancy of your connection. Ideally, you should always have blue bars of only a pixel or two in height. If you are commonly getting big triangles of yellow on the graph, your connection is inconsistant.
In a heavy firefight, it is normal for modem players to see yellow bars in the bottom graph, which should return to green when the action quiets down. If you are getting several red bars visible, you may want to look for a server that drops less packets.
There are a few tuning variables for people trying to optimize their connection:
The most important one is "rate", which is what the connection speed option in the menu sets.
We are fairly conservative with the values we set for the given modem speeds: 2500 for 28.8, 3000 for 33, and 3500 for 56k.
You may actually be connecting faster than that, and modem compression may be buying you something, so you might get a better play experience by increasing the values slightly.
If you connect at 50000 bps, try a rate of 5000, etc.
I err on the conservative side, because too low of a rate will only make the movement of other things in the world choppy, while too high of a rate can cause huge amounts of lag.
Note that the optimal rate will be somewhat lower than a rate for QW or Q2, because I now include the UDP packet header length in the bandwidth estimate.
You can ask for a different number of snapshots by changing the "snaps" variable, but there isn't a lot of benefit to that. Dedicated servers run at 40hz, so stick to divisors of that: 40, 20 (default), 10. A snaps of 40 will usually just cause you to hit your rate limit a lot faster. It may be usefull for tuning rate, if nothing else.
You can adjust the local timing point with "cg_timenudge ", which effectively adds local lag to try to make sure you interpolate instead of extrapolate. If you really want to play on a server that is dropping a ton of packets, a timenudge of 100 or so might make the game smoother.
------
One more addition to net cvars:
"cl_maxpackets" will restrict the maximum number of outgoing packets to prevent client to server rate problems. This does not limit the client framerate. This defaults to 20, which might actually be a bit low. You might try experimenting with raising this to 40.
"cl_maxfps" still exists, but it will never need to be used for networking reasons.
-------
* converted cvar allocation to indexes to allow range checking
* cgame converted over to use vmCvar_t instead of cvar_t needed for interpreted cgame
* fixed server crashing string bug
* adjusted scoreboard for 8 players
* show hostname on connection screen
* fixed null model warning on startup
* more space for hostname on local servers screen
* fixed mac Open Transport memory buffer bug, this was causing most of the mac crashes
* made Info_ValueForKey() case insensitive
* sv_privateClients, sv_privatePassword. this allows you to reserve slots on a public server for password access while allowing most to be freely available
* "server is full" message on connect screen
* archive handicap in config file
* cheat protect r_nocurves
* byte order independent zip checksum
* removed cl_stereo, use glConfig.stereoEnabled
-----------------------------------------
John Carmack's .plan for Apr 27, 1999
-----------------------------------------
* cgame converted to use local buffer based lerpTag for interpretation
* cgame converted to use local buffer based argv for interpretation
* new sound code to remove latency
* added drop shadow to field characters and fixed scrolling
* fixed edge-of-bounce-pad misprediction error (server side)
* remove broken weapon-stay dmflag
* made menu gfx never picmip
* cheat protect r_lightmap
* clear sound buffer before any file IO
* use GetOSEvent instead of WaitNextEvent on mac when fullscreen. removes hitches caused by other tasks and gives a performance boost
* continuous scoreboard / ping update when tab is down
* put version number on menu background
* fixed toggle cvar bug
* dim out behind floating menus
-----------------------------------------
John Carmack's .plan for Apr 28, 1999
-----------------------------------------
* converted sound positioning away from callback method
* increased mac memory zone by 5 megs
* new memory allocator for temporary render use during init
* converted cmodel references to handles and range checked
* converted sound references to handles and range checked
* converted file references to handles and range checked
-----------------------------------------
John Carmack's .plan for Apr 29, 1999
-----------------------------------------
* rework versioning for architecture tracking
* use a seperate endpoint for address resolves on mac
* hide OTLook warnings if "developer" isn't set
* defered mac renderer scanning until after mode set so 8 bit desktops don't confuse it
* global motd/update server
* fixed view model animations on models with custom anims
technical note:
Q3 can run networked player movement in either an asynchronous or synchronous manner. The default mode is to allow all client movement to be done asynchronously with the servers advancement of time.
The primary reason is to allow player movement to be predicted on the client side. The primary drawback is that while your movement is smooth, the other players that you see running around in the world move with a jerkiness that is relative to their framerate and network connection quality. It is NOT necessarily relative to their ping - a player on a fast system with a clean modem connection can move smoothly. If you see a player stuttering around, either they have a bad framerate, or the network connection between them and the server or you and the server is poor. The amount of stuttering is sort of the sum of the dropped or variable packets on BOTH connections.
You can force Q3 to run all clients synchronously by setting "g_synchronousClients 1" on the server. This will make Q3 behave similar to Q1 for networking. All movement will be lagged except view angles, which are still short-circuited immediately.
Some people claim to prefer synchronous movement when everyone had a very good ping, but I don't personally think it is ever a play benefit. It makes leading players a bit easier, but I think the crisp movement control of client side prediction is a much better tradeoff.
However, there is still a reason for using it: recorded demos come out a LOT smoother looking when running with sync. Note that q3test does not allow demo recording and playback, so this is just for future reference...
-----------------------------------------
John Carmack's .plan for Apr 30, 1999
-----------------------------------------
I put together a document on optimizing OpenGL drivers for Q3 that should be helpfull to the various linux 3D teams.
http://www.quake3arena.com/news/glopt.html
-------
* vmtest framework, q3asm work
* converted scene building to procedural style. allows better error checking, better performance characteristics when interpreted, and is a setup stage for SMP optimizations in the renderer if I get around to it
* protected some potential div by 0 areas in cgame
-----------------------------------------
John Carmack's .plan for May 04, 1999
-----------------------------------------
* seeded random numbers differently on tourney restarts
* fixed events on initial snapshots
* removed g_maxentities configuration, set by G_ENTITY_BITS
* cl_motd 0 to allow never sending request packets
* fixed map cache clearing bug
* cg_drawFPS 1 for running fps counter in corner
* remove all teleport destination pads
* moved checkmap out of cgame
* moved time positioning out of cgame
* made usercmd overrun freeze in place instead of snapping back
* slightly increased shotgun spread
* protected against using a cleared clientinfo
* use snapped origin from players for linking to prevent slight prediction errors during player collisions
-----------------------------------------
John Carmack's .plan for May 05, 1999
-----------------------------------------
* client side predict item pickups. running over items was one of the few remaining locally perceived signs of lag
* new pont-in-patch test code
* fixed pathname errors when mac users had slashes in their paths: "B/W mac". sigh.
-----------------------------------------
John Carmack's .plan for May 07, 1999
-----------------------------------------
* changed grabbed items to SVF_NOCLIENT instead of EF_NODRAW now that the pickup event is on the player
* clear event flags with event on reset
* move playerstate to netfield bit communication
* fixed configstring delta sequencing issue after initial gamestate
* extended the netgraph: short red lines are missing client to server packets (need to drop 3 in a row)
* extended cg_debugevents
* increased cl_maxpackets to 30
* fixed bug with console field not getting drawwidth set
* fastsky implies noportals
* changed fastsky color
* q3map now fixes tjunctions at fog boundaries
* build optimized tree with visible hulls of sides
* adjusted plane culling to avoid some cracks
* r_facePlaneCull
* fixed too-lax colinear collapse to avoid some cracks
-----------------------------------------
John Carmack's .plan for May 08, 1999
-----------------------------------------
There is one must-fix issue and a couple smaller issues remaining before the release candidate build, then we have to do a lot of testing on it. I made a lot of significant changes in the last week, and I'm sure there are some things we still need to sort out before we inflict it on the general public.
We are aiming for sunday, but understand that that means sunday evening, not sunday morning.
If saturday night / sunday morning testing on the release candidate turns up significant problems, we will put off the release until they are fixed. That could be later sunday night, or it might not make it until monday night.
The previous release delays for win32 were issues out of our control, but this release rests squarely on me. The content and other issues are ready, but we still need to make sure all the new code is solid.
* fixed give item bug
* new first snapshot timing
* moved sun drawing outside of sky shader to fix showtris
* r_drawSun
* handle all shader tesselations in q3map with tjunc fixups
* different flatness epsilons for edge vs border grids
* reorganize sound directories
* removed footsteps on non-metalic and non-water surfaces
* fixed bug with multiple looping sounds
* client side predict teleporters, go to "hyperspace"
* precache remaining liquid sounds
* don't fire jumppad events if upward velocity
-----------------------------------------
John Carmack's .plan for May 09, 1999
-----------------------------------------
We would up making tweaks to both maps today, so the data didn't reach final form until a few hours ago.
I just finished making release candidates for all three architectures, but I already found a couple problems that need to be fixed.
If everything goes perfectly (ha), and I nail these problems immediately when I wake up, then we might make it out tonight, but it is looking a bit doubtful.
There are a few known issues that I decided NOT to hold the test up for:
The gauntlet is functioning correctly, but the visuals are wrong. The designed behavior is that when you hold down attack it will scan for a target and only punch forward when it hits. The visuals currently show it punching constantly.
Dynamic lighting is currently taking a really excessive amount of cpu time. If you are having performance problems in firefights, you may want to turn it off. The option is in the preferences, or you can just issue "r_dynamicLighting 0" at the console.
The powerup item sounds aren't global across the entire world since I went to the client side predicted items.
There are some cases when a weapon that was picked up with a predicted item and immediately fired doesn't make a muzzle flash.
* fixed fs_copyfiles after ospath split
* fixed look-at-killer
* changed railgun impact to plasma dish
* convert connect packet to infostring
* put footsteps back in...
* r_drawsun 0 by default to avoid probs for now
* fixed event clear overwrite problem
* client side predict weapon switch on item pickup
* changed sound fallbacks to "visor" from "male"
* made turbulent texcoords based off of xyz instead of st
-----------------------------------------
John Carmack's .plan for May 10, 1999
-----------------------------------------
A good day of work. I just finished a long test game with all three architectures, and everything looks solid.
As far as I can tell, these are ready to go after making installers and such, but everyone else has an oportunity to find bugs while I sleep...
We won't hold up for minor gameplay issues, but if anyone turns up a repeatable crasher I will rebuild everything.
Barring problems, we should start rolling the releases out tonight.
* fixed crash case on fallback from an unsupported fullscreen
* fixed overrun with very fast system connecting to a very lagged server
* fixed bad Z_Free on sounds not found
* fixed autoswitch with sync clients
* fixed losing console field on positive histories
* fixed demo recording and playback with new net code
* handle signed bit fields in msg code
* fixed playerstate event bit loss on encoding
* fixed "bad clientnum on player entity"
* reenabled corpses sinking into ground
-----------------------------------------
John Carmack's .plan for May 11, 1999
-----------------------------------------
I am offering a bounty for server crashing bugs. Q2 had several releases forced out because of malicious attacks on all the public servers, so I want to try and flush out what I can during Q3's testing phase.
There is a server running in the debugger here at crashtest.idsoftware.com (192.246.40.68). Anyone that can repeatably hang or crash this system can have a $100 prize and some misc bit of Q3A paraphenalia that I can dig up.
Operating system level attacks don't count -- only things that I can actually fix or protect against in my code.
Denial of service attacks don't count if they require upkeep, but if there is a fire-and-forget DOS attack, it will still count.
Any actions you can perform with the released client are fair game. Crashing the client isn't good for a bounty, but I would still like to know about it.
Custom attack programs are also fair game. These are actually what I am most concerned about -- malicious programs that goes through and crash all listed servers.
Ideally, you would practice on a private server under your control and only hit crashtest when you think you can repeat it.
If you find one, email me the instructions so I can reproduce it. Include "CRASHTEST" in the subject so I won't miss it.
First come, first served, one bounty per bug. I will update crashtest with our internal builds, so it will certainly be possible that an attack on the released servers no longer functions on crashtest.
-------------
Now that the first win32 test is out, here is The Plan for going forward:
All future releases should be same-day for all architectures.
There may be an exe-only update to the current distributions if there are significant problems, but it isn't scheduled.
The next major test release will include a new one on one map designed for tournement play, and new executables with server and game modifications, but will not require downloading a new pak0.pk3.
The release after that will introduce various teamplay rules on the original two maps. This version will likely be another full download, because I know that I still have a couple things to change in the map format. This will probably be the first test running with the virtual machine.
The final major test release will introduce the single player game with bots and ranks.
After any bugs are shaken out of that, it will be the "Q3 Demo" instead of the "Q3 Test", and we should be ready to release the full game to stores.
In an ideal world, people that aren't prepared to deal with in-development software would wait until then to form an opinion of the product.
-------
Everyone should realize that many popular net links are going to be clogged up with q3test downloads for a while, so net play may be a bit patchy to a lot of servers.
-------
BigImp wins the first prize. It doesn't crash the server, but fmtspec names will crash all clients that try to log on. Technically that would be an upkeep required DOS attack, but I'll let this one go.
I even had a "FIXME: make vsprintf safe" comment by the offending line...
I am going to update the server to filter out all % chars that come in over the net to prevent any other similar things.
------
Sami Tammilehto wins the second prize. Some large connectionless packets can cause crashes.
This one was a result of me having the maximum token size defined lower than the maximum string size.
-------
Do NOT send bug reports and game comments directly to me! If I have to filter through hundreds of emails a day, I won't get any more work done... Only crashtest related problems should come to me, everything else should go to q3feedback@idsoftware.com.
-------
You can bias the level of detail lower than allowed in the menu with "r_lodbias 2", which will force all models to the lowest lod. The view weapon will look very ugly.
Another little speedup option that isn't offered in the menus is: "cg_simpleitems 1" this removes the extra rings and spheres around some items.
You can also turn off all the gibs with "cg_gibs 0".
* clear game memory at init, which fixes the stuck-at-intermission problem on mac servers
* fixed mismatched free / Z_Free in demo menu
* removed unused reference to sprites/plama.md3
* automatically get sounds from model name
* scale sensitivity by zoom
* immediately archive changes to latched cvars
* cheat protect r_portalonly
* don't print "XXX connected" on level restarts
* fixed "give item" on levels where 0,0,0 is in solid
* fixed timedemo
* don't play pain falling sound if dead
* fixed falling damage sound not snd specific
* fixed crashtest 2
* fixed crashtest 1
* q3map_backshader
* q3map_globaltexture
-----------------------------------------
John Carmack's .plan for May 12, 1999
-----------------------------------------
We had to upgrade the crashtest machine to NT sp 5, because some people were attacking it with windows crashers. Those don't count.
Crashtest #3 from [iBO]QWhAX0R was a combination of two problems:
The symptom was disconnecting all clients with an illegible server message. This turned out to be caused by the fact that I was parsing strings out of my net buffers with a MSG_ReadChar() function, and I was checking for EOF as a negative one return value. I had to change this to a MSG_ReadByte() call, because -1 was showing up in the messages, which then caused a parse error because it wasn't really the end of the message.
The actual root of that issue was code like this:
[code]
{
char buffer[MAX_STRING_CHARS];
...
strncpy( buffer, input, sizeof(buffer) - 1 );
...
}
[/code]
No buffer overruns are possible, but buffer is not forced to be zero terminated if on the stack. I'm pretty sure this was a result of copy-and-paste code where buffer used to be a static with a guaranteed zero, but it made me find several other places where similar things were happening.
I had started using a Q_strncpyz() function a while ago that guarantees a trailing zero and doesn't require the -1, but it turned out that between code I had written a long time ago, and code that either Cash or Brian had added, there were still a lot of normal strncpy calls around. A lot of them were wrong, too. Either missing the -1, or missing the dedicated 0 fill in.
Crashtest #4 from Jim Paris was a variation on the first part of #3.
Only one of these attacks so far has been a server crasher, but I have been giving the bounty for anything that immediately kicks all the players. I probably won't give it for attacks that only overflow some lagged clients, but I'll evaluate as they happen.
I am off to E3 now for a bunch of PR silliness, so if crashtest goes down, it won't be back up for a while...
* fixed crashtest 4
* fixed crashtest 3
* fixed jumping-over-item pickup prediction error
* made "Couldn't load sound" a developer only warning
* fixed demo recording not including portal surface entities
* precache grenade bounce sounds
-----------------------------------------
John Carmack's .plan for May 19, 1999
-----------------------------------------
Now that all the E3 stuff is done with, I can get back to work...
I was stuck in a room the entire time doing press interviews, but it seemed to have gone well. It was mentioned to me that there were a few people on the show floor with forged badges that read "John Carmack -- Id Software". As if forged email / irc / icq isn't enough of a problem. Sigh.
The "download" crashtest was first reported by Rick Hammerstone. That was a pure dumbass mistake on my part.
I should be sending the accumulated crashtest bounties off tomorrow.
The plan right now is to have an update release next week that will have lots of bug fixes and cheat protections, but not too many new user visible features.
I finally got around to implementing dual processor acceleration today. I still have a couple issues to resolve and some more rearranging to do, but it is giving 20%+ speedup right now in a worst-case situation for it.
When completed, I expect the average speedup to be in the 40% to 80% range, depending on what is going on and the video configuration. Scenes with lots of dynamic lighting and lots of sounds and other client processing going will show the largest speedups. It helps the slow scenes more than the fast scenes, which is basically what you want.
I am going to shake this out with the Windows (NT) code first, but it should definately make its way to the linux port eventually.
I know SMP is a que for all the BeOS folks to ask about ports, so I'm going to head that off: Be has all the code for Q3 (and Q2, for that matter), and a version of Q3test should be available by the time they ship a release OS with OpenGL hardware acceleration.
There will probably also be an SGI-irix port.
Regarding both of those ports: they are not supported ports, and will be maintained by volenteers (like the current MacOS X port). Update releases will lag the official ones, and we won't committ to ANY dates.
I am doing all of my development on intergraph and sgi-NT hardware, but when I have everything rock solid, I will give Nvidia and ATI's NT drivers a try. I would be somewhat shocked if they didn't explode -- I doubt multiple threads playing occasional tag team on a context has been well tested.
True, only a tiny fraction of our players (probably less than 1%) will be able to take advantage of this, but I consider SMP usage to be an important technology to nurture over the coming years.
The top of the benchmark chart should be an SMP system (assuming the NT drivers have all the optimizations of the '98 drivers), and it will also be possible to build a reletively cheap SMP system (say, dual 400's) that outperforms the best single processor system.
-----------------------------------------
John Carmack's .plan for May 22, 1999
-----------------------------------------
The SMP support is solid enough to play with now. The only feature that is still broken is light flares.
As a happy consequence, some of the cleanup work I did for SMP gave a couple percent speedup even when running without the separate thread.
On my development system, a dual 300 mhz intergraph realizm II, the low res timedemo scores went from 27.8 to 37.8 with "r_smp 1". This is only a 35% average speedup, but at some times (lots of dynamic lights in complex scenes) the speedup is 90%+. Gameplay is noticably smoother.
The rendering thread is almost always the blocking factor, so the faster the card and OpenGL driver, the larger the speedup will be.
This is explicitly a two thread producer / consumer, so there is no benefit to more than two processors. The app is well behaved, using sleeping syncronization so that you usually still have half a processor free for other operating system functions.
Hopefully we will be able to test with some fast consumer cards sometime soon.
------
A lot of people asked what was done differently this time vs the last time I tried (without benefit) to use SMP.
My original attempt was to make a DLL that intercepted all OpenGL calls and let a separate processor execute them. The benefit would have been that all OpenGL applications could have gone faster. The problem was that the bandwidth required to encode all the commands was enough that the processor overhead was as much as it would have taken to just do the geometry on the main processor.
It would have still been a win if the geometry side was doing lots of work, like multiple lights, user clip planes, and texgens, but for the vast majority of geometry, it didn't balance out. If someone wanted to try that using the PIII or AltiVec streaming memory operations, it could probably still work.
The current SMP code is implemented directly into the renderer, and a lot of things were moved around and double buffered to allow it to use data in place, instead of having to copy it off.
------
Some people expressed surprise that Quake3 wasn't threaded already.
Threading has been presented so often as the "high tech" "cool" way to program, that many people aren't aware of the downsides.
A multi-threaded program will always have somewhat lower throughput when running on a single CPU than a single threaded program that polls in explicit places. The cost of a context switch at the cpu level is negligible, but the damage that it can do to the cache hierarchy can add up to a noticeable amount in bad cases.
The much larger problem is that you lose tight control over when things occur. If the framerates are low enough, it isn't a huge issue, but for applications running at 30+ fps, you really don't want to trust the OS scheduler to coordinate multiple threads and have them all get in every frame. Yes, with explicit sleep() calls you can sort of get it working, but at that point, you might as well not be using threads.
A good example of not-quite-in-sync issues in the windows mouse performance. A PS/2 mouse only samples 40 times a second, so when you get an app updating at around that speed, you will get 0/1/2 scheduling variances.
They are also not terribly portable, and a pain in the ass to debug.
-----------------------------------------
John Carmack's .plan for May 26, 1999
-----------------------------------------
* basic joystick controls, some work still needed for advanced controlers
* r_dlightBacksides 0 option
* forced cvar_restart when version changes
* fixed some flare-in-fog problems
* fixed skin color in menus
* print obituary message even when you are the killer, so all kills get an entry in the logfile
* fixed bugs in line token parsing when quotes or commands aren't white space separated
* multiprocessor acceleration "r_smp 1"
* increase menu dimming
* increased rocket damage radius from 120 to 150 units
* check for running server in all server commands (dumpuser, etc)
* new cvar cheat setup -- by default, only archived variables can be changed when not cheating
* "cg_drawstatus 0" only removes status bar
* "cg_draw2d 0" removes all 2d
-----------------------------------------
John Carmack's .plan for May 27, 1999
-----------------------------------------
* enable scissor test properly
* archive r_lodBias
* cg_draw3dIcons 0 option
* data cheating protection
* userinfo renamed to clientinfo, added state and current server address
* don't forward commands to a server when playing demos
* fixed NULL extension on dir command
* added one more shotgun pellet
* added CG_Shutdown for cgame cleanup
* fixed jitter in rising smoke
* increase minimum time before reusing an entity slot
* soundinfo reports current background streaming file
* changed IPX separator to . from :, moved port processing to system independant code
* auto port scan wasn't updating the net_port cvar
* attack button presses reset inactivity timer now
* increased the forced respawn time from 10 to 20 seconds
* show smp on gfxinfo, slight reformat
-----------------------------------------
John Carmack's .plan for May 30, 1999
-----------------------------------------
For the past couple of weeks, I have been spending some development time on linux, and for the first time on a non-NEXTSTEP unix platform, I have actually been enjoying it.
While Id has been supporting linux since the Doom days, I have not personally been much of a linux user -- it was always ddt or zoid doing the actual coding and testing. Every year or so I would install a linux distribution and play around with it for a few days, but I would always leave feeling that it was still pretty crude (UI wise) compared to the NEXTSTEP UI I was used to, or even what I had used on other commercial unix workstations and windows.
There have always been a ton of reasons to like linux, but the user interface was enough of an issue that I couldn't buy into it completely.
The gnome user environment in Red Hat 6.0 is finally at a level that I consider it a valid alternative to commercial desktop environments. Overall, its still not as smooth, consistant, or complete as windows or the mac, but is does have its strong points, and things seem to be progressing quite rapidly.
Its still not something you would give to a purely casual computer user, but I won't be surprised if even that changes in a couple years.
CodeWarrior for linux is also a significant aspect of my enjoyment. Its a sort of crappy 1.0 port with a lot of little issues, but the editor works well enough, which is the important thing for me. I have never been able to stand vi or emacs for long enough to become proficient in them.
The code that I have been playing with most is the matrox g200 GLX driver.
Matrox is the first of the major 3D chip vendors that has had the guts to publicly release register level documentation for their 3D chips.
An accelerated X windows OpenGL driver has been put together with this by building on top of the existing Mesa and GLX projects.
It actually runs quake, quake2, and q3test. It doesn't run them FAST, but the quality is good, and I am impressed nonetheless. It is bordering on playable with all quality options set to the minimum on a fast computer, but it still has a ways to go before casual users should take a look at it.
It is steadily improving, and I hope Matrox will be pleased enough with the progress that they will release the documentation for their setup engine to go with the rasterizer.
In testing q3 on it, I noticed that with picmip set to 0, textures would get corrupted and it would never settle on a working set. The current Apple OpenGL drivers also have exactly this problem.
The cool part is that this driver is completely open source. I downloaded the project code, browsed through it a bit, and changed two lines of code to fix the bug. That RULES.
The next thing is sort of funny. I had been suspecting that a lot of the OpenGL drivers were clearing the entire depth buffer for each of the 3D icons on the status bar, instead of limiting it to the scissor region. I added code to the g200 driver to explicitly crop the clear region in the driver, but it never got executed. A little more investigation showed that I had been making an improper assumption for years -- scissor is not enabled by default. Doh.
Ever since noticing that glquake cleared the screen borders when the view is sized down, I had been operating under the assumption that intergraph just had a bug in their drivers. I had double checked that glClear was supposed to be limited to the scissor region, so I thought they were just messing it up.
Now I know that I was just being an idiot about that for the last three years... With scissor enabled, most of the cards got a few percent faster.
* dynamic curve level of detail. r_subdivisions determines the maximum level of detail, r_lodCurveError determines how quickly polygons are pulled out with distance
* devmap sets cheats 1, map sets cheats 0
* change weapon item upscale to 1.5 instead of 2
* always toss items forward, even if looking up or down
* draw ammo in grey while weapons are reloading
* change railgun shader while reloading
* fixed head models not showing proper skin
* skip all shell eject code when cg_brassTime 0
* fixed sound memory overallocation
* profiling and rearrangement
* fixed dead spectator bug
-----------------------------------------
John Carmack's .plan for Jun 03, 1999
-----------------------------------------
Whee! Lots of hate mail from strafe-jupers!
Some reasonable messages have convinced me that a single immediate jump after landing may be important to gameplay. I'll experiment with it.
Strafe jumping is an exploitable bug. Just because people have practiced hard to allow themselves to take advantage of it does not justify it's existance. When I tried fixing the code so that it just didn't work, I thought it changed the normal running movement in an unfortunate way.
In the absense of powerups or level features (wind tunnels, jump pads, etc), the game characters are supposed to be badasses with big guns. Arnold Schwartzenegger and Sigourney Weaver don't get down a hallway by hopping like a bunny rabbit.
This is personal preference, but when I play online, I enjoy it more when people are running around dodging, rather than hopping.
My personal preference just counts a lot. :-)
btw, here are the current weapon effects:
gauntlet: 50 pts, 400 msec / punch
machinegun: 10 pts, 100 msec / shot
shotgun: 11 pellets of 10 each, 1000 msec / shot
rocket launcher: 100 pts direct hit, or 100 pts splash damage falling off
over 120 world units, 800 msec / shot
plasma gun: 20 pts direct hit or 15 pts splash damage over 15 units,
100 msec / shot
railgun: 100 pts, 1500 msec / shot
lightning gun: 8 pts, 33 msec / trace, max range 768 units
grenade launcher: 100 pts direct hit, or 100 pts splash over 150 units,
800 msec / shot.
bfg: 40 pts instant splash damage over 100 units, 100 msec / shot
flamethrower: to be determined, but short range / wide angle
Splash damage is calculated from the edge of the player's box, unlike
quake1, where it was calculated froom the player's origin.
* ignore cl_maxpackets on LAN
* changed cl_packetdup to 1 by default, and archived
* defaulted com_maxfps to 100 and archived, automatically disabling during timedemo. It was possible to lag out some client connections on ultra fast systems even with cl_maxpackets set fairly low due to a huge number of individual commands being created
* 250 msec minimum time between landing and jumping again. I hate having players bouncing around all the time...
* fixed bug with large r_picmip values (white shotgun sight bug)
* new lightning and rail beam drawing
* player torso twitches with pain sounds
* r_drawstrips changed to r_primitives and archived, with changes: default "0" uses glDrawElements if compiled vertex arrays are present, or strips of glArrayElement if not. "1" forces strips, "2" forces drawElements, "-1" skips drawing
* increased rocket speed to 900 from 800. decreased direct hit damage from 120 to 100. splash damage same as 1.05
* removed sound-in-use dialog, auto skip after second try
* made userinfo persistant on server across level changes
* allow different servers to respond to a challenge, allowing redirecting server proxies
* notice bad ip addresses for connection: 192.1234.123.122
* removed neck length pivot to prevent view poking into low subdivided curves. Also make aiming when looking up or down more precise.
-----------------------------------------
John Carmack's .plan for Jun 27, 1999
-----------------------------------------
For the past couple years when talking to chip makers about new 3D features, 3D texture maps would always come up, and I would have to mutter something like: "I think it is a good thing, but I can't give you a really awesome example of using it. 3D Noise functions. Stuff like that."
I have the awesome example now: Lighting.
Through the entire development of Q3, I have been wanting to do dynamic lighting of the world differently. Many of the efficiencies gained by Q3's new map format result in larger and larger lightmaps while it reduces polygon count. That meant that even small dynamic lights may force a lot of work to generate new lighting textures.
I had been wanting to find a way to trade off additional rendering passes for less CPU time.
This is a perfect example of me outthinking myself.
I knew it would have to be something with projecting a light spot texture onto the world geometry, but I am keenly enough aware of the issues and limitations in projecting a 2D texture onto 3D geometry that my mind kept looking at worst case scenarios -- how do you project a continuous texture onto the inside of a dome with the proper intensities, and some situations with curves. I thought it would require some non-trivial per triangle analysis and operations, when I really just wanted something that could be done on a per-vertex level.
Recently, the 3D texture insight hit me.
Create a 3D texture of, say, 32*32*32 texels and fill it with your light attenuation function, bright point in the center, fading off to black at the edge. Set it to clamp mode, so any values beyond the edges stay black. Set up a texture coordinate generation matrix to position the light relative to your models. Its a simple translate and scale. Set up lighting calculation to only generate a value based on the relative angles, ignoring attenuation. Use the resulting modulated texture mapping as your "lightmap" to modulate another texture pass, or add to a previous lighting pass.
This has several important benefits:
It looks good (and identical) on both small and large polygons. Vertex based lighting needs an appalling amount of tessellation to avoid visibly triangulation artifacts. Even if you tessellate to the same sample density as your lightmap grid (hundreds of thousands per map), vertex lighting still looks worse, because it is a triangular instead of bilinear interpolation.
It can be done completely by upcoming lighting and texgen hardware, but is efficient in software implementations.
It can be shadowed by either stencil volume shadows or shadow buffer tests.
Post Q3A, I plan on doing a research engine that is fully dynamically lit and shadowed instead of using lightmaps.
Ok, but we don't have 3D textures in any hardware now, so this doesn't help me for Q3A.
I continued thinking along complex lines, like "Any planar slice of a spherical 3D texture will be a 2D circle, and the three triangle points can occupy any location on their respective rings".
It still looked like a big mess.
I got tired of thinking about it, and just started coding in some infrastructure to do extra lighting passes. I just made a gradient circle texture for the lighting, and generated the texture coordinates by just translating and scaling X and Y from the world coordinates.
I got it running, grabbed a rocket launcher, and fired down a hallway. It looked just fine. I was stunned. I had been thinking about complex ramifications of weird edge cases for the past year when all it took was a couple hours of programming and the simplest possible approach to make it work decent. Sigh. :-)
The only real addition required was a fading of the light contribution with Z distance from the surface. The downside to this hack is that while you get a nice ball of light moving over floors and ceilings, you only get a wash of light on walls. In hindsight, I can analyze this in the context of our games and say "Almost all movement and targeting takes place in the XY plane in a FPS, so accuracy along the Z axis is not necessary."
The somewhat more obvious change to dynamic lighting that wasn't related to the projection issue is that I have it currently set up as a postprocess pass, rather than an additive pass on the lightmap. Adding to the lightmap is more "right", but it really complicates the multitexture implementation, and on some surfaces, the lightmap is actually done after the base texture, so it couldn't be added to. The end result is that brighter areas are changed more by dlights than dark areas. I don't feel TOO bad about that, because its not like the dlight is much of a proper lighting simulation even in the best case...
* fixed tourney restart
* fixed jittering on plats
* fixed ref use without a world
* new default image that lets you see mapping coordinates
* fixed reliable sequences on restarts and demos
* allow maxclients to change between levels
* randomize shell ejection start position and angles
* display attackers head after every wound
* added gamma and overbright support to mac version
* removed table from sound mixing
* remove smoke and blood puffs when you run through them
* set cheats to 1 on disconnect
* shader sort value is now floating point
* new trajectory type "TR_INTERPOLATE", skip interpoaltion for other types
* fixed door open timing
* fixed bug with >8 portal areas
* added area print to r_shwocluster
* removed all the partial shader match cruft, fixing a crash-on-load
* fixed bug with personal shadows not being setup properly
* nomipmap shader parm split and expanded:
nopicmip : ignores r_picmip so image will always be high res
nomipmap : forces a single level texture, used for console font
* shrank sound mixing buffer for better caching
* new shader option: q3map_lightsubdivide. larger values make q3map -light proceed faster
* new vertex array interleaving
* fixed bmodels not counting patches in bounds
* fixed patch sphere culling on rotating entities
* cg_simpleItems now draws sprite items
* fixed serverid being 0 when map is started from cmdline
* fixed UI on mode changes
* replaced 1280*960 mode with 1280*1024
* test all rotated orders for tristrips from faces
* fixed RB_SurfacePolychain to not duplicate vertexes
* changed planar face surface type from a convex polygon to a general collection of coplanar triangles
* change renderer to use bmodel surface list instead of tree
* changed areabits pointer to areamask array
* fixed bad loop when client reliable message overflows
* fixed swapinterval after vid_restart
* wall mark clipping moved out of cgame, extended for better wrapping over multiple brushes
* picmip defaults to 1 under all cases now
* automatic curve LOD grouping
* duplicated SCR_ functions into cgame
* implemented Micahel Julier's optimization work
* implimented Michael Gold's SMP patch
* new reliable command transport
* use ENTITYNUM_NONE and ENTITYNUM_WORLD constants, which are now in MAX_GENTITIES range for safe net transport
* wait for attack released after respawning before firing
* added a delay before moving everyone to intermission spot
* fixed bug with regibbing of gibbed body ques
* blood trails behind gibs
* changed localents over to trajectories
* removed world as entity zero, clients are now 0 to MAXCLIENTS-1
* changed game interface functions to use clientnums instead of pointers
* removed speculative usercmd_t from user packets
* new item pickup code, fixed silent item pickup on grazing hits, and expanded the pickup range by 20% without changing the physical bounding box
* allow a single "quick jump" without delay
-----------------------------------------
John Carmack's .plan for Jul 03, 1999
-----------------------------------------
AMD K7 cpus are very fast.
Some timedemo numbers (a new demo, not comparable to previous scores):
Run at 640*480*16 bit color to emphasise the cpu/driver performance rather than the hardware fill rate.
[code]
K7-600 K7-550 PIII-500
TNT2 ultra 16 bit 73.9 68.5 53.8
Voodoo3 3000 16 bit 70.5 65.2 46.0
[/code]
This is with K7 optimized drivers vs seperate PIII optimized drivers.
There is still wiggle room there in that it is possible that more effort was expended to make the AMD drivers perform better. That is perfectly valid from a consumer's point of view, but muddies the technical CPU comparison.
On identical code run on the systems, there was some more interesting data:
On my map processing tools, the K7 was faster than the PIII, but only slightly more so than the reletive clock rate increase. I would guess that this is due to larger data sets that don't fit in cache as well.
On the matrox OpenGL drivers, which have not been optimized very much and (to my knowledge) contain no PIII specific code, the K7 was a LOT faster.
The bottom line is that I feel comfortable standing behind the statement that the K7 is faster than the PIII. I will have to wait for some stuff to come out of NDA to provide a more detailed technical analysis.
Architectural cleverness is all well and good, but if AMD can't keep the clock speed up with intel, they will still fall behind. A 700 mhz PIII would probably find a lot of applications (especially integer apps) where it would outperform a 600 mhz K7.
* stabilized cg_showfps
* added append support to module file opening
* automatic logging of game scores to games.log
* fixed guantlet firing action
* force a vid_restart on WM_DISPLAYCHANGE messages
* fixed sticking on stairs on very fast framerates
* fixed sticking on stair when jumping
* fixed sticking in corner while falling physics bug
* fixed slide down steep slope physics bug
* r_showimages texture use debugging tool
* cg_freezeDemo cvar
* cg_drawSnapshot cvar
* fixed warnings after demo playback
* changed "stopdemo" to "stoprecord"
* fixed phantom windows on task bar after exit
* check for unset player animation
* fixed the snap-down-look-up bug with very high sensitivities
* reduce inflicted damage by handicap
* all pmove results as events for proper demo playback
-----------------------------------------
John Carmack's .plan for Jul 24, 1999
-----------------------------------------
I was in San Jose for the past week. The original idea was to go into "hermit mode" holed up in a hotel room and get a lot of work done without any phone calls, email, or coworkers to distract me, but I wound up with meetings scheduled every day with various valley companies. Next time I want to try that, I'll pick some state like Montana... :-)
The top priority was getting the virtual machine done, but I hoped to also get some more of the map data optimizations completed. I definately underestimated how big of a hole daily meetings would punch in the amount of work I could accomplish.
On wednesday I was sweating a bit, not sure if I would have the VM finished in time, but it all came together in the last two days.
The virtual machine interpreter is now completely functional, and cgame can switch between being loaded as a binary .dll or an interpreted .qvm at the change of a cvar.
The basic setup is that I have a modified version of the lcc compiler that generates assembly files that are processed by a new tool "q3asm" into a .qvm file that can be interpreted by q3. You can still use normal dll's for debugging, then release interpreted code. You can release a binary dll if you need some native system services (some networking stuff, for example) or are doing very compute intensive work, but I strongly encourage everyone to try to use the virtual machine.
The interpreted code has two prime benefits: portability and security.
User mods done with qvms will automatically work on mac, linux, and any other oddball ports that get released.
A qvm program cannot modify anything outside its private address space, and the "system calls" to the game strictly limit what can be touched. I was scared about the binary dll's in Q2, but rationalized it to myself that people running public servers should be aware of the dangers. With Q3 allowing client side programming, it just needs to be safe for everyone.
Packaging is also improved somewhat, because the virtual machine programs can be included in pak files and are automatically handled through the search path.
Unfortunately, even after doing most of the straightforward optimizations, the interpreter is causing a 20% slowdown in timedemos right now.
I am pretty committed to running cgame interpreted, but if I don't get a significant speedup, we may have to leave the server side game module as a native dll. The ui module can obviously be interpreted.
There are lots of paths I can take to get the performance up:
Write more efficient cgame code. I will definately be looking at this. A lot of things that just didn't matter at all when in native code now add up enough that they should be fixed. I want to avoid flexibility tradeoffs if possible.
Move more calculation from the cgame to system calls. I have already done this for the obvious things like memset, matrixmultiply, etc. Any future work will involve restructuring cgame code to focus lots of work into simple stateless functions. The trick is to leave all the flexibility in the cgame while moving the work.
Implement LCC compiler optimizations. I doubt I will pursue this, unless there are existing optimization projects based on LCC with minimal backend changes required.
Implement more complex interpreted instructions and peephole combine the bytecodes into the new instructions. I think this has some promise.
Assembly code the interpreter loop. The compiler is generating good code, but there is still room for gain. I don't want to pursue this until the high level optimizations are wrung out.
Load time compilation from bytecode to native code. This would bloat a lot, especially on RISC cpus. I'm not sure that a straightforward implementation would have significant speed benefits over an assembly interpreter loop, and I won't have time for writing an aggressive native optimizer.
Deal with the performance cost and optimize the renderer some more to compensate.
-----------------------------------------
John Carmack's .plan for Jul 29, 1999
-----------------------------------------
* log all client transitions, item pickups, and kills
* changed joystick axis to act just like arrow keys so they can be bound in the controls menu for strafing. Yes, this does remove slow walking from joystick movement, but it makes everything a lot cleaner.
* fix up PantherXL trackball support
* removed bad clear command from dedicated console clear button
* tournement queuing of spectators to enter the game
* track wins and losses as long as you stay on a tourney server
* spectators are now in fly mode instead of noclip, and use teleporters
* pass serverTime instead of msec for command timing, prevents timescale cheating
* track dual eventParms on player state
* draw crosshair and name in spectator mode
* fixed rcon
* r_colorMipLevels texture visualization tool
* don't allow weapon select and cycle when in follow mode
* archive cl_yawspeed and cl_pitchspeed
* don't draw place line on scoreboard when spectating
* fixed console chatting during intermission
* better recursive error handling
* fixed curve surface flags (no impact when landing on a curve bug)
-----------------------------------------
John Carmack's .plan for Jul 30, 1999
-----------------------------------------
I apologize for not getting the Mac version released this last time.
All conspiracy theories aside, here is what actually happened:
I had my San Jose travel scheduled a while ago, and we were expecting to have the release done before I left. We didn't quite make it, and nobody else at the office knew how to build the mac version after Cash made some last minute changes.
When I got back, Graeme was taking off to move his family down here. Graeme is in charge of building all the installs for our releases.
I considered just tossing new executables with the latest fixes for everything, but some data has changed, and it just isn't worth the hassle right now.
We will be making proper new releases for all architectures monday night when he gets back.
* fixed marks fading properly in fog volumes
* show weapon in fov >90, adjusting position down as needed
* allow run/bob variables to be changed in non-cheat games
* update scoreboard info while at intermission
* fixed angles on resetplayerentities, corrects twitch on players as you come through a teleporter or respawn
* print "waiting to play" for tourney spectators
* fixed tied rank with 0 score and spectators
* return to roaming spectator when a followed client quits or spectates
* release windows cursor when running windowed and the console is down
-----------------------------------------
John Carmack's .plan for Aug 01, 1999
-----------------------------------------
* changed RF_PORTALSURFACE to an entity type
* add sprite indexes in strip order
* changed builtin models to refentity types
* prevent respawn until resting on ground
* fixed unecessary dlight shader checking
* no splashe when lg hitting sky
* removed cvar_restart on version change, caused problems with dedicated server startup
* fixed stupid error that made skies double render
* fixed stupid error that made skies draw an uneeded bakground pass
* support add-mode multitexture for skies
-----------------------------------------
John Carmack's .plan for Aug 16, 1999
-----------------------------------------
As I mentioned at quakecon, I decided to go ahead and try a dynamic code generator to speed up the game interpreters. I was uneasy about it, but the current performance was far enough off of my targets that I didn't see any other way.
After getting over being sick the start of the week (someone from QC must have brought me a flu present), I decided to dive in to it.
At first, I was surprised at how quickly it was going. The first day, I worked out my system calling conventions and execution environment and implemented enough opcode translations to get "hello world" executing.
The second day I just plowed through opcode translations, tediously generating a lot of code like this:
[code]
case OP_NEGI:
EmitString( "F7 1F" ); // neg dword ptr [edi]
break;
case OP_ADD:
EmitString( "8B 07" ); // mov eax, dword ptr [edi]
EmitString( "01 47 FC" ); // add dword ptr [edi-4],eax
EmitString( "83 EF 04" ); // sub edi,4
break;
case OP_SUB:
EmitString( "8B 07" ); // mov eax, dword ptr [edi]
EmitString( "29 47 FC" ); // sub dword ptr [edi-4],eax
EmitString( "83 EF 04" ); // sub edi,4
break;
case OP_DIVI:
EmitString( "8B 47 FC" ); // mov eax,dword ptr [edi-4]
EmitString( "99" ); // cdq
EmitString( "F7 3F" ); // idiv dword ptr [edi]
EmitString( "89 47 FC" ); // mov dword ptr [edi-4],eax
EmitString( "83 EF 04" ); // sub edi,4
break;
[/code]
(yes, I could save a few bytes in those opcodes by moving the sub edi, but I am trying to leave the subs at the bottom and the adds at the top, in case I want to add a peephole optimizer)
I was writing test programs as I went, so I thought it was still going quite well. I quit for the day with only six opcodes left to write.
Today I got in, wrote the last opcodes, and started running the full cgame module.
The first problem was obvious: the loading screen graphics came up with the default image instead of the text font. I quickly found and fixed a problem with system call return values.
It was then able to get into the game, but the FOV was clamped out to 160, and it crashed when you fired your gun. That turned out to be my failure to fix the operand stack correctly in the rarely used structure copy opcode.
The next problem was a little more distressing. You could run around the level, but all the items had an X coordinate of 0. This took a little while to find, and turned out to be a twitchy case of sometimes getting an extra value on the operand stack.
At that point everything looked like it was working perfectly. I was ecstatic. I started running timedemos to see what the performance looked like. Then I ran a demo without timedemo on, and near the end of the demo it crashed. Then I found out that if you play in a level for a few minutes, it either gets an error of some kind, or crashes.
I was quite unhappy about that. Debugging a non-deterministic crash in generated code. Joy.
What I wound up eventually doing was to make gigantic log files of all system call interactions and compare the compiled code against identical runs with the interpreter. I had to tweak a few things to make the processing exactly the same between them, but it let me find where things first started to diverge. It turns out I was letting the top of my compiled code's local stack creep down a bit with each call. Let it run long enough, and it started hitting important things. If I had logged stack pointers instead of parameter values, I would have found it a whole lot quicker...
Now, I am pretty confident that it is correct.
The generated code is pretty grim if you look at it, in part due to the security measures (mask and add for each load/store), and in part due to the fact that it is a straight bytecode translation:
[code]
06214DD0 83 C7 04 add edi,4
06214DD3 C7 07 00 4D 0A 00 mov dword ptr [edi],0A4D00h
06214DD9 8B 1F mov ebx,dword ptr [edi]
06214DDB 81 E3 FF FF 1F 00 and ebx,1FFFFFh
06214DE1 8B 83 30 00 DC 05 mov eax,dword ptr [ebx+5DC0030h]
06214DE7 89 07 mov dword ptr [edi],eax
06214DE9 83 C7 04 add edi,4
06214DEC C7 07 2C 00 00 00 mov dword ptr [edi],2Ch
06214DF2 8B 07 mov eax,dword ptr [edi]
06214DF4 01 47 FC add dword ptr [edi-4],eax
06214DF7 83 EF 04 sub edi,4
06214DFA 8B 1F mov ebx,dword ptr [edi]
06214DFC 81 E3 FF FF 1F 00 and ebx,1FFFFFh
06214E02 8B 83 30 00 DC 05 mov eax,dword ptr [ebx+5DC0030h]
06214E08 89 07 mov dword ptr [edi],eax
06214E0A 8B 07 mov eax,dword ptr [edi]
06214E0C 29 47 FC sub dword ptr [edi-4],eax
06214E0F 83 EF 04 sub edi,4
06214E12 83 C7 04 add edi,4
06214E15 C7 07 40 00 00 00 mov dword ptr [edi],40h
[/code]
Code bulk is also up there, at about 5x the bytecode version. There is definately some savings to be had with better opcode selection, but no more than 30% or so at best.
Performance is within my tolerance now:
Q3demo1 dll: 52.9
Compiled: 50.2
Interpreted: 43.9
Q3demo2 dll: 50.1
Compiled: 46.5
Interpreted: 38.7
I will probably work a bit more on performance, but that is the ballpark that it will be in. 5% speed hit in most levels, somewhat more in the big open arenas. Next week I will be getting the other modules set up for running in the virtual machine and see how their performance is.
It is a pretty cool setup - you can have some modules as dlls, some as interpreted bytecodes, and some as compiled bytecodes. We will leave the user interface interpreted to save memory.
Tomorrow I am going to get all the byte order issues worked out for powerPC. The interpreter doesn't even work there yet because of inline constant byte order issues. Fixing that will slow the interpreter a little more, but that shouldn't be any problem, with the performance oriented stuff being compiled.
Doing the PPC compiler will be a bit messier because the tools aren't as nice, and the fact that it will involve a whole lot of Mac crash/reboot cycles before it stabilizes, but I think I know what to watch out for now.
VC6 crashed on me about a dozen times in the last few days, probably due to my having show-code-bytes on in the dissassembly window, but it was still pretty damn useful through the whole process.
I am curious to see how the RISC code bulk turns out. The instructions are going to be longer, but all the constants can be held in registers. Should be interesting.
I don't think I am going to be in any hurry to do MIPS/ALPHA/SPARC code generators. One or two code generators and execution environments is educational, but that will be more than enough for me. If we do port to other architectures, they can still run with the interpreter or binary modules until someone else gets up the inclination to do a code generator.
-----------------------------------------
John Carmack's .plan for Aug 26, 1999
-----------------------------------------
The current plan is that we will have another test release around the middle of next month. This version will be running game/cgame/ui with the virtual machine, and will include single player play against bots. No new maps.
I will be releasing the source for all the VM modules with it, and setting the executable up so that it will allow modified modules to be used.
The modified LCC and q3asm will be available both in source form and precompiled, so a professional development environment will not be requried. Using MSDEV to debug binary dll's does make exploration a lot easier than adding prints to interpreted code...
Some minor porting work on the tools will be necessary to do development under linux. The effort would be greater for mac development, because the tools are inherently command line based.
The map editor and tools will not be released until after the game hits store shelves. To be completely clear: you are not legally licensed to create or use addon maps with the test.
I am hoping that this public review will turn up bugs before we complete the game. 50k lines of code is quite a bit to go over, but people familiar with previous games will have a good head start in the game module.
The best possible situation would be if exploration of the code evolves into a tier system, with either moderated or limited access lists that I can follow without being swamped. I can't afford to be too involved in helping everyone figure out the basics. I have plenty of confidence in the mod communities ability to work that out. :-)
---------
Some people have been mistaking memory swapping in 1.08 for network problems.
We did controlled, back to back tests against the previous versions, and the networking is identical if you have enough memory.
The addition of the new character model and all the new menu code and graphics has caused the game to begin to have some swapping problems on 64mb machines if you have all the quality options up high or are running other things.
I am looking into what I can do to reduce memory consumption for the next release, but in the meantime, you can turn down sound quality, geometry, or texture detail to get rid of it.
If you have less than 64mb, go buy more memory! The final game will have an option to run in less memory, but the graphics and sound quality will be a lot lower.
----------
I made a simple change in the file management that I think is clearly a big win.
Instead of scanning for pak0.pk3, pak1.pk3, ... pak9.pk3 in game directories, I now scan the entire directory for all .pk3 files, and add them to the search path in alphabetical order.
This gives us the same needed functionality we have now -- overriding things in pak0.pk3 with updated data in pak1.pk3, but it also gives a signficant benefit to the user community.
There has been a lot of requests to have textures inside maps like Quake1 did, but I was not willing to do that. Having the files separate saves an immense amount of duplication, and keeps the internal architecture uniform.
Now, you can just add all your new art into a pk3 (zip) file along with your map and users can just drop that single file into their quake3/baseq3 directory as a single operation.
This will also be nice for custom models, which require several component parts: legs, torso, head, animations, levels of detail, and skins.
So, if you are strictly ADDING media (models, textures, maps, etc), then you can just drop the pk3 files in the normal directory.
If you are REPLACING data (code modules, menus, etc), you should make a new game directory and put the pk3 there. Starting up with "quake3 +set game mygame" will make all the mygame pk3 files override anything in baseq3. You could do it by just naming your pk3 file "zzzstuff.pk3", but then you wouldn't have a way to run the game without the addons.
A prudent person might choose to put ALL addons into a separate directory and leave baseq3 pristine for official additions.
----------
Other stuff that has been done lately:
* don't clamp dedicated server or client times until much later -- prevents time resets under ordinary conditions
* fixed CG_ProcessSnapshots: cg.snap->serverTime
* only drift time on receipt of packets with a steady ping -- improve catchup after drops
* fixed players getting stuck together
* new pak file support
* derez mac resource file
* intro/loop option for music files
* fixed lerp-through-world when changing teams
* show ping and netgraph for spectators
* fixed timescale off by one problem
* tracked down player count wrong on server list
* info_spectator_start entity
* shader language change: clampTexCoord removed, added clampmap
* r_debugSort cvar for working on transparency sorting problems
* changed minimum cl_maxpackets to 15
* fixed can't-respawn-when-someone-is-on-your-body
* fixed dlighting over alpha tested surfaces
* z_stats lists all blocks >= given size
* fixed wasted model slots
* increased com_hunkmegs
* com_buildScript cvar to force loading all media and quit on errors
* fixed bad playerstate interpolation across teleporters
* converted local sounds to sfxHandle_t
* new fog code doesn't require subdivisions
* fixed sun positioning problem
* added fogging of triangle objects
* fixed devmap issue
* make g_log a filename instead of a 0/1
* g_logsync option to force a flush after each write must be set at time of log file creation
-----------------------------------------
John Carmack's .plan for Aug 28, 1999
-----------------------------------------
* spinning machinegun barrel
* changed q3data -origin option to -offset, defaulted to 0 0 24 for all player grabs
* removed second parm from -lod in q3data
* fixed 0 ping on last player killed before fraglimit
* better ping calculation right after transitions
* add time back to scoreboard
* sv_maxRate option to force all clients to play with a max rate. This can be used to limit the advantage of LPB, or to cap bandwidth utilization for a server. Note that rate is ignored for clients that are on the same LAN.
* fixed bad name vs name in tourney after first player left
* added hitch warning messages to server console
* new time clamping rules for net play
* avoid sending usercmds during connection
* send explicit heartbeats to the master server when a server transitions to or from empty or full
* shaders that aren't found will return index 0, but still keep the allocated slot to prevent rescanning if registered again
* use nextSnap for player prediction when available
* removed teleport dest invisible objects
* reduced client to server bandwidth by 35%
* changed logging for chats to guarantee parsing properly with names that conflict with commands:
from: G_LogPrintf( "%s say: %s"
to: G_LogPrintf( "say: %s: %s"
-----------------------------------------
John Carmack's .plan for Sep 01, 1999
-----------------------------------------
I have been working on our memory footprint for the past couple days. There are two types of paging that occur:
One time only paging that eventually settles down after you have run around the entire level a few times. This can be a result of having lots of data around that isn't actively used during the game, or was only used at startup.
True capacity misses, where the game is actually touching more memory during play than you have available. This is usually due to there just being too many textures and models.
Loading a large map with full resolution, 32 bit textures and mipmaps consumes almost 40 megs of texture space. The default of picmip 1 and 16 bit textures reduces that to six or seven (not all images observe picmip) in theory, but some OpenGL implementations keep a 32 bit version of the texture around even if the card only uses 16 bit (I consider this inapropriate), so it may still be twelve or more megs just for the images.
I have been able to save a couple megs off of the true capacity requirements by not rolling through some larger buffers when not needed (not smp and not dedicated server), and several megs more of one-time data mostly by moving a lot of static tables to dynamic allocation.
I can probably save another meg of true capacity and I think three or four megs of initialization data. The big thing that I might be forced to do is go to a skeletal animation system. I would hate to do that at this late a point in the project, but it would save about two megs per player model in the game.
In the process of chasing down the static memory hogs, I finally got around to starting something I have needed to do for years: learn perl.
I was scanning through a linker map file looking for large gaps in addresses, thinking to myself "this is one of those things you can probably do in three lines of perl code". I have many and varied excuses for why I have never gotten around to it before, mostly involving the fact that I have C parsing code that lets me get what I need done with only minimal headache when I do force myself to do some text file grovelling.
I decided my excuses weren't good anymore, and went out to the bookstore and grabbed the llama book. Many of you would have been amused seeing me go through the
print "Hello, $name!"
tutorial code as I did the examples in the first couple chapters. :-)
I got my task done, so now I just need to force myself to write little perl programs whenever a need comes up, until I get fluent with it.
* save 2.5 megs by reworking shader allocation
* save 1 meg by not double buffering backend if not smp
* convert all tr. arrays into pointers
* don't allocate as many snapshotentities when non dedicated
* new shader option: deformvertex move
* stackable deformvertex
* reduced lightning damage by 10%
* light emit from two sided surfaces
* reduced starting machinegun ammo in teamplay to 50 from 100
-----------------------------------------
John Carmack's .plan for Sep 02, 1999
-----------------------------------------
I have been getting a lot of requests for commentary on two subjects lately:
Nvidia's new geometry accelerated card with the funny name.
It is fast. Very, very fast. It has the highest fill rate of any card we have ever tested, has improved image quality over TNT2, and it gives timedemo scores 40% faster than the next closest score with extremely raw beta drivers.
The throughput will definately improve even more as their drivers mature.
For max framerates in OpenGL games, this card is going to be very hard to beat.
Q3's target of about 10,000 triangles a frame doesn't stress this card at all. If you want more polygons out of Q3, you can do:
r_lodBias -2 // don't use lower detail models
r_subdivisions 1 // lots more triangles in curves
r_lodCurveError 10000 // don't drop curve rows for a long time
I haven't looked at the stencil shadow stuff in a long time, but it gives the largest increase in triangle use (and a lot of fill rate as well):
cg_shadows 2 // turn on stencil shadows (if you have a stencil buffer)
Apple's new G4 systems.
The initial systems are just G4 processors in basically the same systems as the current G3. There will be some speedup in the normal C code from the faster floating point unit, and the Apple OpenGL has AltiVec optimizations, so framerates will improve somewhat. The limiting factor is going to be the fill rate on the rage128 and the bandwidth of the 66mhz pci bus and processor to main memory writes.
The later G4 systems with the new memory controller and AGP will have better performance, but probably still limited by the new 3D card.
After Apple gets all their driver tuning done, it will be interesting to try running timedemos at low resolution to factor the fill rate out. Apple has a shot at having the best non-geometry accelerated throughput, but it will still be tough to overcome a K7 with an extra hundred or so mhz.
On a purely technical note, AltiVec is more flexible for computation than intel or AMD's extensions (trinary ops), but intel style write combining is better for filling command buffers than the G4's memory streaming operations.
-----------------------------------------
John Carmack's .plan for Sep 06, 1999
-----------------------------------------
It looks like we are going to go to a skeletal model system. Jim Dose of Ritual had already started on an exporter from character studio, so we decided to just meet in the middle.
I implemented the loading and rendering support this weekend and tested it with a couple hand-inserted bones, so now we just need to write the glue between character studio and the new .md4 format.
The new format is bone based, but it is NOT hierarchial. Each vertex just has an arbitrary weighted list of the bones that influence it. Bones are just 4x3 matricies of floats.
A hierarchial skeleton has some advantages (angles instead of matricies, ability to do IK, etc), but this is a direct and simple replacement for our existing infrastructure that doesn't require any cached state per model instance.
A single .md4 file holds multiple level of detail surface sets, which all share the same bone frames.
In use, it is exactly like the existing models (interpolate between two frame numbers), it just saves a huge amount of space.
I used perl to generate my test data, and it was definately faster than having a separate msdev open and doing it in C.
I am trying to use parenthesis on all perl functions, but when I type "print", my fingers seem to have a flashback to applesoft basic fifteen years ago, and I wind up with bare quotes on prints and parens on everything else...
Does anyone know if there is an existing msdev syntax coloring file for perl? (no, I don't want to switch to a different editor!)
* md4 model loading and displaying
* removed clip models from cgame, use renderer models instead
* fixed mover pushing again
* fixed bug with culling of mirrors made of multiple faces
* fixed quad on spinning machinegun
* surfaceparm alphashadow - This causes q3map -light to check individual texture pixels on transparant surfaces for light shadowing instead of making the entire surface either cast or not cast shadows
-----------------------------------------
John Carmack's .plan for Sep 28, 1999
-----------------------------------------
Ok, obviously we didn't get a release out in the middle of the month...
We are still hashing out the single player game, so it still isn't immediately immenent.
* weapon switch animations at 20hz instead of 15hz, cuts switch time from 600 msec to 450 msec
* initial spawn flag for single player
* finished new fog code
* fixed walking underwater friction problems
* autosprite2 now selects the longest axis to pivot on, and allows any texture mapping, not just unity
* fixed autosprite on entities
* fixed lurching during low timescale
* reduced machinegun damage
* set clamp mode on 2D pics
* take hostname off of single player connect
* remvoed dlighting on skies and solid lava
* fixed lower body twitch when copytobodyque with a motion
* always show your score in second box if not in first
* sarge as default model
* com_blood 0 option for no gibs and no blood on hits
* mouse click aborts cinematic
* show tourney scores in all games, add fraglimit
* removed tripple bunny-hop protection, it was too arbitrary and didn't accomplish it's goal
* pump event loop during level loading
* added pass count to shaderlist
* default to CGEN_IDENTITY / CGEN_IDENTITY_LIGHTING based on blendSrc
* optional simplified blendfuncs: blendfunc
* new shader command: deformVertexes normal
* new shader command: tcgen vector ( ) ( )
* fixed fog on alpha tested surfaces
* reduced com_maxfps to 85
* defined shaders for menu and console backgrounds
* reset players on clientinfo transitions
* windows icons
* fixed powerups on spinning barrels
* removed some latency from lightning endpoint
* fixed lightning bolt drawing too far
* removed color clamping from entity lighting
* moved all 2D drawing to shader pipeline
* r_printShaders tool
* moved dlighting into world node descent
* pause when menu is up in single player
* fixed double EV_FIRE_WEAPON
* r_singleShader optimization tool
* some renderer optimizations
* better multitexture collapsing
-----------------------------------------
John Carmack's .plan for Sep 29, 1999
-----------------------------------------
I wrote this in answer to a question on the mac opengl programming list, but it is of general enough interest to programmers that I am repeating it here.
>You also mentioned display lists... Can you explain what some of the
>major things that should be rendered using a display list are? I see
>the importance of using them for characters (animation) and objects, but
>what about using them for the rest of the world, particles, and other
>things that are nifty.
This is not yet a big issue, although even pure software OpenGL's could perform some optimizations with display lists that aren't possible with vertex arrays. With hardware geometry acceleration, it can be an honest 4x improvement in throughput.
The important point is that once geometry acceleration becomes a primary target, practically everything will have to be rendered with display lists, or you will run into a nasty case of Amdahl's law.
In a busy Q3 battle, the triangle count may be split roughly evenly between character models and world geometry. Going from an empty scene to a pitched battle can result in a 50% performance drop if you are triangle limited. Not great, but livable.
If we kept the same ratios and designed for geometry acceleration with all the static world geometry in display lists, then the empty scene could have 4x the geometry and still be running the same speed. However, current OpenGL display lists can't really accelerate high quality skinned characters, so when an equal number of character polygons was in scene and passed through normal direct rendering, the performance would drop to 20% of the original. Unacceptable.
So, either you would have to use significantly different polygon counts in characters and the world, or some new API features would need to be defined. Nvidia has a skinning extension that gives some benefit, but still requires a character to be broken up into one static list per bone pair, instead of a single list for the entire character.
Rendering a few thousand particles or other procedurally generated triangles directly isn't going to be a big issue, but the bulk of the work is going to move towards static vertex data.
My advice for display lists is to use them for just raw vertex/color/texcoord data, and keep your state changes done with direct commands. This allows you to still sort display lists to minimze state changes, and prevents drivers from ever having to check state internally. Some hardware architectures can nicely encapsulate all state changes in a single dma buffer, but register sharing among different fields sometimes requires the driver to do manual masks, negating much of the async display list benefits. Texture swapping also complicates state changes inside display lists.
You want to make the display lists as big as practical, but there is a tradeoff betwen culling tightness and display list size.
-----------------------------------------
John Carmack's .plan for Oct 01, 1999
-----------------------------------------
* 10 crosshairs to select from (cg_drawCrosshair 1 - 10)
* cg_crosshairx / cg_crosshairy adjustment. I'm not convinced these are a good thing, because the crosshair is accurate in Q3 at the default position (unlike Q2, which had an offset firing position)
* more packet encryption
* join as spectator in all team games
* cg_predictItems 0 option to not do local prediction of item pickup
* rank players counting ties, so the third player when the lead is tied is third place, not second
* properly stack all status elements in upper right and lower right corners so they can all be visible
* faced q3map problem giving black corners in vertex light
* fixed +button4 not causing footsteps
* fixed bad groundplane clipping on angled jumppads
* show "snc" on lagometer when g_syncronousClients
* new shader command: rgbgen const ( )
* new shader command: agen const
* snap dropped item positions
* randomized offsets of bubbles
* fixed cgame restarts processing snapshots from 0
* proper setup for external/predictable player events
* fixed q3map vertex lighting bug after alphashadows
* do personal pain sounds on health transitions so they can never be missed
* timeout clear player events
* fixed hang in UI_ProportionalStringWidth
* quad event implicit on weapon fire
* fixed gamestate not retransmitting bug
* fixed timeout issue when paused
* subdivide command times if <15fps, fixing low com_maxfps physics exploits
* fixed footsteps playing when walking backwards
* new options at start of animation.cfg file: footsteps, headoffset
-----------------------------------------
John Carmack's .plan for Oct 04, 1999
-----------------------------------------
* allow cg_thirdPerson and cg_thirdPersonRange in games
* added execed .cfg files to journal file
* single pass plasma explosion effect to save overdraw
* rescaled and sized gib and mark blood to save a lot of overdraw
* fixed cg.time < snapshot time on vid_restart
* cl_showSend network debugging tool
* back to requiring a before commands on the console to distinguish them from chat messages. Tab completion automatically adds the slash.
* new tab command completion with complete to longest common, multiple match listing, complete first token, etc
* separate version check for game/cgame in addition to system
* show r_finish in gfxinfo
* never show self as attacker icon
* callvote vote - Caller automatically votes yes vote has a 30 second timeout each client can only call 3 votes a level vote is displayed on screen with totals
* renamed cg_gun to cg_drawGun
* box cull triangle soup models
-----------------------------------------
John Carmack's .plan for Oct 05, 1999
-----------------------------------------
* fixed steady snapshot test
* fixed incorrect 0 ping if past client messages
* fixed loser-disconnecting-at-tourney-intermission sorting problem
* general purpose flood protection, limiting all user commands to one a second by stalling the client, so the commands don't actually get dropped, but are delayed as needed
* replace headnode overflow with lastCluster
* fixed bad extrapolation on unpausing
* fixed player twitch on unpausing
* print client names on loading screen
-----------------------------------------
John Carmack's .plan for Oct 07, 1999
-----------------------------------------
* r_primitives 3 path for non-vertex array testing
* specify sex in model animation.cfg file
* proper dropping of failed bot inits
* removed identical pain sounds
* serverTime strictly increasing across levels
* added GL_DECAL multitexture collapse
* windowed mouse on mac
* fixed byte order issue with curve clipping on mac
* made com_defaultextension buffer safe
* fixed levelshot and added antialiasing to image
* don't clear bot names before kick message
* made servercommand sequences strictly increasing across level changes
* unpause on vid_restart
-----------------------------------------
John Carmack's .plan for Oct 11, 1999
-----------------------------------------
* handle window close events properly
* enable r_displayRefresh selection and feedback on mac
* avoid AGEN_IDENTITY when CGEN_DIFFUSE_LIGHTING
* colorized railgun muzzle flash, impact flash, and mark
* exactly synced animating textures with waveforms and collapsed all explosion sequences into single shaders
* removed unneeded black pass on hell skies
* fixed grenades sticking to steep slopes
* scan for next highest fullscreen resolution when exact mode fails (fixes SGI flat panel failing to init)
* all cgame cvars now have a cg_ prefix (crosshair, fov, etc)
* clear clientinfo before parsing configstring
* make all feedback voiceovers share the same channel
* fixed nodraw curves
* fixed obits from shooter entities
* fixed chat char issue
* separate gentity_t fields into sharedEntity_t
* reintegrated q_mathsys with q_math
* cg_forcemodel also forces player sounds
* unknown cmd commands don't chat
* fixed strip order on text quads
-----------------------------------------
John Carmack's .plan for Oct 14, 1999
-----------------------------------------
* make sure video is shutfown for errors on startup
* automatic fallback to vm if dll load fails
* compressed jump tables for qvm
* removed common qfiles.h and surfaceflags.h from utils and game
* don't load qvm symbols if not running with developer
* "quake3 safe" will run the game without loading q3config.cfg
* ignore network packets when in single player mode
* dedicated server memory optimizations. Tips:
com_hunkMegs 4
sv_maxclients 3
bot_enable 0
* fixed logfile on mac
* new time drifting code
* fixed file handle leak with compressed pk3 files
* q3data changed to remove shader references from player models
* throw a fatal error if drop errors are streaming in
* fixed com_hunkMegs as command line parm
* spawn spectators at intermission point (info_spectator_start has been removed)
* new sound channel for local sounds
* fixed follow toggle on bots
* don't write to games.log in single player
* fixed improper case sensitivity in S_FindName
-----------------------------------------
John Carmack's .plan for Oct 17, 1999
-----------------------------------------
The next release will be the full "demo" release for quake 3. It will include bots and a new, simple level that is suitable for complete beginners to play, as well as the existing q3test maps.
The timing just didn't work out right for another test before we complete the game.
We plan on releasing the demo after code freeze, when the entire game is in final testing, which will give us a few days time to fix any last minute problems that show up before golden master.
No, I don't have an actual date when that will be.
-
I got an iBook in on friday. It is sort of neat (I need to go buy an AirPort, then it will definately be neat), but it is currently way too slow to play Q3 on.
Apple's high end G3 and G4 systems with rage128/rage128pro cards and latest drivers are just about as fast as equivelant wintel systems, but the rest of the product line is still suffering a noticable speed disadvantage.
The new iMac has a rage128, but it is only an 8mb/64bit version. Still, with agp texturing it is a solid platform with performance that is certainly good enough to play the game well on.
Existing iMacs have 6mb ragePro video. ATI's windows drivers for the pro have come a long ways, and Q3 is plenty playbale on windows with a rage pro (ugly, but playable). On apple systems (iMacs and beige G3's), the performance is sometimes as low as HALF the windows platform. The lack of AGP contributes some to this, but it is mostly just a case of the drivers not being optimzed yet. The focus has been on the rage128 so far.
The iBook is also ragePro based, but it is an ultra-cheap 32 bit version. It does texture over AGP, but it is slooooow. I suspect it is still driver issues, because it is still very slow even at 320x240, so that leaves hope that it can be improved.
Another issue with the Apple systems is that Apple 16 bit color is actually 15 bit. I never used to think it made much difference, but I have been doing a lot of side by side comparying, and it does turn out to be a noticable loss of quality.
* new lg splash
* added channel number for local sounds so feedbacks don't override announcers
* removed scoreup feedback sound
* expand score tabs as needed for large scores
* fixed bfg obit
* fixed swimming off floors
* fixed swim animation when jumping in shallow water
* fixed first weapopn switch problem
* convert all joystick axis to button events (twist is now bindable)
-----------------------------------------
John Carmack's .plan for Oct 23, 1999
-----------------------------------------
An announcement:
We have hired Robert Duffy as a full time employee, starting in December.
He has been maintaining the level editor since the release of Q2, but a number of things have convinced me it is time to have a full time tool-guy at id.
The original QE4 editor was my very first ever Win32 program, and while Robert has done a good job of keeping it on life support through necessary extensions and evolutions, it really is past its designated lifespan.
I want to take a shot at making the level editor cross platform, so it can be used on linux and mac systems. I'm not exactly confident that it will work out, but I want to try.
Many of the content creation tasks seem to be fairly stabilized over the last several years, and our next product is going to be pretty content directed, so I can justify more engineering effort on writing better tools.
It is time for a re-think of the relationships between editor, utilities, and game. I am still an opponent of in-game editors, but I want to rearrange a lot of things so that some subsystems can be shared.
All of that added up to more than I was going to be able to do in the time left after the various research, graphics, and networking things I want to pursue.
* added r_speeds timing info to cinematic texture uploads
* fixed loop sounds on movers
* new bfg sound
* "sv_pure 1" as default, requires clients to only get data from pk3 files the server is using
* fixed fog pass on inside of textured fog surfaces
* properly fog sprites
* graphics for scoreboard headers
* show colored names on attacker display and scoreboard
* made "no such frame" a developer only warning
* count a disconnect while losing in tournement mode as a win for the other player
* fixed running with jump held down runs slow
* draw "connection problems" in center of screen in addition to phone jack icon
* cut marks properly on non-convex faces
* fixed bug with command completion partial match and case sensitivity
* fixed console dropping on level start
* fixed frags left feedback after restarts
* fog after dlight
* removed fogged stages from shader, dynamically generate
* removed fogonly shader keyword, infer from surfaceparm
* removed uneeded reinit of shader system on vid_restart
-----------------------------------------
John Carmack's .plan for Nov 01, 1999
-----------------------------------------
* play chat sound during votes
* draw 2D icon for ammo if cg_draw3dicons 0
* fixed losing input on menu vid_restart
* made "vote" and "callvote" completable
* remove mac about dialog
* fixed demos skipping inital time due to loading
* fixed timing on timedemo startup
* don't flash attacker when damaging self
* display capturelimit instead of fraglimit on score tabs in ctf
* recursive mirror/portal changed to a developer warning
* fixed bug with follow mode spectators
* battle suit shader
* notsingle spawn option
* separated torso and legs priority animation coutners so gesture doesn't mess with legs
* adjusted value to prevent missed launch sound on accelerator pads
* setviewpos x y z yaw, same parms as viewpos command
* stop sound on file access
* fixed developer prints from renderer
* defered client media loading, only load models and sounds for new players when you die or bring up the scoreboard
* fix for double colliding against same plane and getting stuck
* dropped LG damage to 160 pts / sec
* don't snap predicted player entity, smooths deaths and movers
* all sine movers are instant kill on block
* fixed items riding on bobbers
-----------------------------------------
John Carmack's .plan for Nov 02, 1999
-----------------------------------------
* remove grapple from give all
* fixed pick-up-two-healths-even-if-you-don't-need-the-second bug
* moved wrap/clamp to image generation function and added to imagelist, fixed an improper clamp on macs
* different menuback for ragepro
* fixed mac button problems with OS9 and wheel mice
* teamplay rule mods:
less MG damage (5 instead of 7)
weapons always have a full load of ammo
don't drop powerups
* changed low detail r_subdivisions to 25 to prevent poke through
* removed warning on empty servercommand when systeminfo changes
* g_debugDamage
* when a vote is tied with all votes in, immediately fail it
* haste smoke puffs
-----------------------------------------
John Carmack's .plan for Nov 03, 1999
-----------------------------------------
* cd check in single player
* removed drop shadow on console input line
* swapped mynx pain sounds
* fixed cleared music buffer on track loop again
* force skins on spectators in team games to prevent having a default waste memory
* only defer to a model with same team skin
* fixed grenade-disappearing-at-floor bug when about to explode
* draw reward medals during gameplay
* added "humiliation" feedback for gauntlet kills
* spread respawn times to prevent pattern running:
[code]
#define RESPAWN_ARMOR 25
#define RESPAWN_TEAM_WEAPON 30
#define RESPAWN_HEALTH 35
#define RESPAWN_AMMO 40
#define RESPAWN_HOLDABLE 60
#define RESPAWN_MEGAHEALTH 120
#define RESPAWN_POWERUP 120
[/code]
-----------------------------------------
John Carmack's .plan for Nov 05, 1999
-----------------------------------------
* check for bad weapon number in non-3d ammo icon on death
* fixed plane catagorization
* error command does an ERR_DROP if given a parm
* don't load high LOD models if r_lodbias
* nobots/nohumans options for player spawn spots
* prevent voice rewards on frag that enters intermission
* dissallow native dll loading if sv_pure
* loaddefered cgame command, issued on addbot
* drop the weapon you are changing TO if you only had a MG or gauntlet
* fixed bounce pad event prediction for all angles
* allow empty tokens in map files
* fixed infos exceeded warning on bot parse
* warning on mismatched mipmap/picmip/wrapclamp image reuse
* fixed pain echo sound after predicted falling damage
* move sound to hunk
* move vm to hunk
* restart game vm in place for map_restarts
* avoid all lightmaps entirely when using vertex light
* pretouch all images after registration
* pretouch all known in-use memory before starting a game
* on error, shutdown client before server, to be more likely to get out of fullscreen before a recursive error
* new pre-allocated memmory manager to crutch up mac
* meminfo command replaces hunk_stats and z_stats
* adjusted scoreboard titles
* no guantlet reward on corpses
* fixed snd_restart when paused
* PRE_RELEASE_DEMO hack
-----------------------------------------
John Carmack's .plan for Nov 09, 1999
-----------------------------------------
* fixed spinning barrel on respawn issue
* clear eflags before intermission
* shutdown menu on starting a cinematic
* mask name colors to 0-7 range
* fixed jpeg loading alpha channel
* try for not-nearest spawn twice instead of once
* made unzoomed exactly identity mouse modifier
* cl_debugmove [1/2]
* m_filter
* fixed time warnings
* allow timelimits to hit with only a single player
* filter local games with different protocol versions
* fixed bad arg 0 after sysinfo configstring
* removed unneeded svc_servercommand at start of command strings
* fixed redundantly loaded level bug
* fixed journal playback from demo build
* removed background image from viewlog window
-----------------------------------------
John Carmack's .plan for Nov 11, 1999
-----------------------------------------
* teamplay menu comment
* shrank and moved "RECORDING demo:" text
* identified and worked around Apple input queue issue
* properly send configstring resets on map_restart
* don't clip sound buffer on file writes
* don't draw scoreboard during warmup
* auto load added bots in single player
* swapped order of map_restart and warmup configstring
* disable dynamic lights on riva128 due to lack of blend mode
* put frags left warning back in all gametypes
* removed joystick button debug prints
-----------------------------------------
John Carmack's .plan for Nov 13, 1999
-----------------------------------------
* graphic for defer
* don't set any systeminfo vars from demos
* A3D fix
* spectator follow clients that disconnect
* stop follow mode before going to intermission so you can ready
* use (fullbright) vertex lighting if the bsp file doesn't have lightmaps
* auto set demo keyword on servers
* finished cd key authorization
* fixed symbol table loading for interpreter
* reconnect command
* removed limit on number of completed commands
* changed default name to "UnnamedPlayer"
* awards over people's heads in multiplayer
* fixed global powerup announcements
-----------------------------------------
John Carmack's .plan for Nov 14, 1999
-----------------------------------------
The demo test is built. It should go up in a couple hours if nothing explodes.
Mac and linux builds won't be out tonight.
* clear SVF_BOT when exiting follow mode
* render temp memory
* new mac GL initialization code
* no zone memory use in music thread
* added check for trash past zone block
* explicitly flush journal data file after a write
* added FS_Flush
-----------------------------------------
John Carmack's .plan for Nov 15, 1999
-----------------------------------------
The way vertex lighting is working in the existing demos is that only two pass shaders (lightmap * texture) were collapsed to a single pass, all other shaders stayed the same.
Xian added some chrome and energy effects to parts of q3tourney2, which changed them from two pass to three pass shaders. We felt that that 50% increase on those polygons was justified in normal play, but as people have pointed out, when you are playing with vertex lighting, that three passes stays three passes instead of collapsing to a single pass, resulting in a 300% increase on those polygons over the way it was before. Still faster than lightmap mode, but a large variance over other parts of the level.
Today I wrote new code to address that, and improve on top of it.
Now when r_vertexlight is on, I force every single shader to a single pass. In the cases where it isn't a simple light*texture case, I try and intelligently pick the most representative pass and do some fixups on the shader modulations.
This works our great, and brings the graphics load down to the minimum we can do with the data sets.
Performance is still going to be down a couple msec a frame due to using dynamic compilation instead of dll's for the cgame, but that is an intentional tradeoff. You can obviously slow things down by running a lot of bots, but that is to be expected.
I am still investigating the high idle dedicated server cpu utilization and a few other issues. The server cpu time will definately be higher than 1.08 due to the dynamic compiler, but again, that is an intentional tradeoff.
A set of go-fast-and-look-ugly options:
[code]
r_mode 2
r_colorbits 16
r_texturemode GL_LINEAR_MIPMAP_NEAREST
r_vertexlighting 1
r_subdivisions 999
r_lodbias 2
cg_gibs 0
cg_draw3dicons 0
cg_brassTime 0
cg_marks 0
cg_shadows 0
cg_simpleitems 1
cg_drawAttacker 0
[/code]
* icons for bot skills on scoreboard
* r_vertexlight is now "force single pass" for all shaders
* modified cd key check to be fire and forget on the client
* file handle debugging info in path command
* network address type of NA_BAD for failed resolves
* better command line variable overriding
* cache scoreboard for two seconds
* sync sound system before starting cinematics
* fixed many escapes disconnect from server exiting the game
* fixed shotgun pellets underwater expending all temp entities
-----------------------------------------
John Carmack's .plan for Nov 16, 1999
-----------------------------------------
The demo servers have general purpose flood-protection that has caused some confusion.
Clients are only allowed to make one command a second of any kind. This prevents excessive flooding by chats, model / name changes, and any other command that could possibly be exploited. The command streams are stalled, so it doesn't have any effect on processing order or reliability.
This means that if you issue two commands immediately after one another, there will be a one second stall before the second command and all movement clears. You see this on the lagometer as yellow spiking up for a second, then dropping away.
Hitting tab for the scoreboard sends a command, so you trigger the flood protection if you bang tab a couple times. This has been fixed so that the scoreboard will never send more than one update request every two seconds, but you will need to watch out for it in the existing demo.
The defered model loading has also caused some confusion, but that is a feature, not a bug. :)
In previous tests, you hitched for a second or two whenever a client with a new model or skin joined a game.
In the demo, when a client joins the game they will be given the model and skin of someone else temporarily, so there is no hitch. The only time it will hitch on entry is if it is a team game and there isn't anyone on the team they join. I make sure the skin color is correct, even if the model isn't.
These "defered" clients will be loaded when you bring up the scoreboard. You can do this directly by hitting tab, or you can have it happen for you when you die.
The point is: you died BEFORE it hitched, not as a result of the hitch.
The scoreboard header is up, but it is still a bit easy to miss.
* fixed high server idle cpu usage (it was spinning in place until maxfps was used!)
* fixed g_password, which is crashing in the demo
* moved svs.snapshotEntities to the hunk
* enable lagometer whenever running a non-local game
* cg_drawTeamOverlay cvar, set to 0 by default
* finished authorize work
* better reporting of unused highwater memory
-----------------------------------------
John Carmack's .plan for Nov 18, 1999
-----------------------------------------
The mac version is out. Go to www.quake3arena.com for links.
The mac version going out has the executable fixes that we have made in the last couple days, but most of the fixes have been in code that runs in the virtual machine, and we can't update that without making it incomptable with the pc version.
The game remains very marginal in playability on 266mhz imacs and iBooks.
A 333mhz imac should be playable for a casual gamer if the graphics options are turned down to the "fastest" setting.
There is still a lot of room for improvement on ATI's side with the RagePro drivers. Almost all the effort so far has been on the Rage128 drivers.
The G3 systems run fine, but a little slower than a pc of equal mhz
The rage128 cards in the G3s are only clocked at 75mhz, so you can't run too high of a resolution, but you can get very nice image quality. I usually play with these settings:
[code]
r_mode 2 // 512*284 res
r_colorbits 32 // 32 bit color
r_texturemode gl_linear_mipmap_linear // trilinear filtering
[/code]
I haven't played on one of the new iMacs or G4's but they both use the rage128 driver, which is fairly high quality now, so they should perform well.
We found a fairly significant problem with inputSprockets and mouse control (motion is dropped after 40msec). I have done a little working around it, so mouse control should be somewhat better in this version, but it will hopefully be fixed properly by Apple in the next IS rev. It isn't an issue if your framerate is high enough, but iMacs never see that framerate on their very best days..
Linux version tomorrow night, if nothing horrible happens.
Some advance warning about something that is sure to stir up some argument:
We should be handing off the masters for all three platforms within a day or two of each other, but they aren't going to show up in stores at the same time. Publishers, distributers, and stores are willing to go out of their way to expedite the arrival of the pc version, but they just won't go to the same amount of trouble for mac and linux boxes.
THE EXECUTABLES FOR ALL PLATFORMS WILL NOT BE AVAILABLE FOR DOWNLOAD UNTIL AFTER CHRISTMAS. This means that if you want to play on the mac or linux, don't pick up a copy of the pc version and expect to download the other executables.
Our first update to the game will be for all platforms, and will allow any version to be converted into any other, but we intend to hold that off for a little while.
We are doing this at the request of the distributors. The fear is that everyone will just grab a windows version, and the separate boxes will be ignored.
A lot of companies are going to be watching the sales figures for the mac and linux versions of Q3 to see if the platforms are actually worth supporting. If everyone bought a windows version and the other boxes sold like crap in comparison, that would be plenty of evidence for most executives to can any cross platform development.
I know there are a lot of people that play in both windows and linux, and this may be a bit of an inconvenience in the short term, but this is an ideal time to cast a vote as a consumer.
Its all the same to Id (I like hybrid CD's), and our continued support of linux and mac (OS X for the next title) is basically a foregone conclusion, but the results will probably influence other companies.
* fixed getting your own dropped / kicked message
* added developer print for all file open write's
* fixed occasional bad color on connecting background
* fixed occasional telefrag at start of skirmish game
* fix not being able to ready at intermission if you were following a bot
* never timelimit during tourney warmup
* fixed local timer on map_restart
* offset sorlag's head model for status bar
* added g_gametype to the votable commands: map, map_restart, kick, g_gametype
* changed sound registration sequence to fix losing default sound
* "sv_floodProtect 0" to turn off flood protection
* converted sequenced messages to static arrays
* fixed custom skin reassignment on all LOD
-----------------------------------------
John Carmack's .plan for Nov 19, 1999
-----------------------------------------
Linux version isn't going to make it tonight. We got too busy with other things. Sorry. Tomorrow.
* shrink zone, grow hunk
* flush memory on an error
* fixed crash pasting from clipboard
* test all compiler optimizations - 5% speedup
* fixed major slowdown in team games with large numbers of players and location markers
-----------------------------------------
John Carmack's .plan for Nov 21, 1999
-----------------------------------------
* fixed not-telefragging bug
* disabled flood protection with local clients
* fixed headoffset and gender on some model changes
* init cg_autoswitch cvars in cl
* fixed clearing of vm bss on restart
* fixed hang when looped part of song isn't found
* fixed two NAT clients connecting to same server
* fixed warning on random once only triggers
* added "g_allowVote 0"
* added developer background sound underrun warning
* move sound loading before clients so low memory defer works across maps
* changed cgame load failure to a drop
-----------------------------------------
John Carmack's .plan for Nov 22, 1999
-----------------------------------------
d:>mkdir research
Ahhhh...
I am very happy with how Q3 turned out. Probably more than any game we have done before, it's final form was very close to its initial envisioning.
I will be getting all the Q3 code we are going to release together over the next week or so. I will write some overview documentation to give a little context, and since you can do game mods without needing a commercial compiler now, I will write a brief step-by-step to modifying the game code.
I'm looking forward to what comes out of the community with Q3.
The rough outline of what I am going to be working on now:
We will be supporting Q3 for quite some time. Any problems we have will get fixed, and some new features my sneak in.
I have two rendering technologies that I intend to write research engines for.
I am going to spend some time on computer vision problems. I think the cheap little web cams have some interesting possibilities.
I am going to explore some possibilities with generalizing 3D game engines into more powerful environments with broader uses. I think that a lot of trends are coming to the point where a "cyberspace" as it is often imagined is begining to be feasible.
I am going to spend more time on some Free Software projects. I have been stealing a few hours here and there to work on the matrox glx project for a while now, and it has been pretty rewarding. People with an interest in the guts of a 3D driver might want to look at the project archives at http://glx.on.openprojects.net/. The web pages aren't very up to date, but the mailing list covers some good techie information.
-----------------------------------------
John Carmack's .plan for Dec 12, 1999
-----------------------------------------
WANTED: Independant OpenGL conformance nazi
I think there is a strong need for a proactive, vendor-neutral OpenGL watchdog, or even a small group, especially in the linux space.
I have been working on the utah-GLX team for quite a while now, and while I have been very pleased with the results, I would like to see more effort spent on doing things as right as possible. Because the developers (me included) are basically just doing the work in their spare time, testing usually only consists of running their favorite OpenGL application, and a few of the mesa demos, or some of the xscreensaver hacks.
Recently I did the initial bringup of a RagePro driver on linux, and I was much more conscious of the large untested feature space, and the tunnel vision I was using to get it to the point of running Q3.
What we need is someone, or a group of someones, who can really exercise different implementations through all corners of the OpenGL specification and provide detailed lists of faults with minimal test cases to reproduce the behavior.
In most cases, the bugs could then be fixed, but even if it is decided that the incorrect behavior is going to stay (to avoid a software fallback in a common accelerated case), there would be clear documentation of it.
I consider performance on the matrox driver right now to be "good enough". There is definately more performance oriented work going on, but given a choice of tasks to work on, I would rather improve quality and coverage instead of kicking a few more fps out of Q3.
One of Alex St. John's valid points was that "The drivers are always broken". There are a lot of factors that contribute to it, including fierce benchmarking competition causing driver writers to do some debatable things and diminish focus on quality. With open source drivers, some of those factors go away. Sure, it is nice to beat windows drivers on some benchmarks, but I wouldn't let pursuit of that goal introduce dumb things into the code.
Some of the windows IHVs have good testing proceedures and high quality drivers, but even there, it would be nice to have someone hounding them about things beyond how well quake releated games run.
The same goes for Apple, especially now that there is both ATI and 3dfx support.
Conformance would be my primary interest, but characterizing the performance of different drivers would also be usefull, especially for edge cases that may or may not be accelerated, like glDrawPixels.
On linux right now, we have:
The traditional fullscreen 3dfx mesa driver
The DRI-GLX based banshee/voodoo3 driver
The utah-GLX matrox G200/G400 driver
The temporary utah-GLX nvidia driver
The newly born utah-GLX ATI Rage Pro driver
If anyone is interested, join the developer list off of: http://glx.on.openprojects.net/
Doing a proper job would require a really good knowledge of the OpenGL specification, and a meticulous style, but it wouldn't require hardcore registers-and-dma driver writing skills, only basic glut programming.
If someone does wind up developing a good suite of tools and procedures and gives one of the drivers a really good set of feedback, I would be happy to provide extra video cards so they could beat up all the implementations.
-----------------------------------------
John Carmack's .plan for Dec 15, 1999
-----------------------------------------
Anna Kang left Id a couple weeks ago to found her own company - Fountainhead Entertainment.
It wasn't generally discussed during her time at Id, but we had been going out when she joined the company, and we were engaged earlier this year. We are getting married next month, and honeymooning in Hawaii. At her thoughtful suggestion, we are shipping a workstation out with us, so I don't fall into some programming-deprivation state. How great is that? :)
Now that Q3A has shipped, the official winner of her Id Software figurine chess set contest is Rowan Crawford for his prose and art.
An honorable mention goes to Reine Hogberg and Peder Hardings for their Q3A Blair Witch Project. They will receive silver Q3A medallions.
-----------------------------------------
John Carmack's .plan for Dec 21, 1999
-----------------------------------------
The Q3 game source code is getting pushed back a bit because we had to do some rearranging in the current codebase to facilitate the release, and we don't want to release in-progress code before the official binary point release.
We still have a Christmas present for the coders, though:
http://www.idsoftware.com/q1source/
Happy holidays!
-----------------------------------------
John Carmack's .plan for Dec 25, 1999
-----------------------------------------
There are a number of people upset about the Quake 1 source code release, because it is allowing cheating in existing games.
There will be a sorting out period as people figure out what directions the Quake1 world is going to go in with the new capabilities, but it will still be possible to have cheat free games after a few things get worked out.
Here's what needs to be done:
You have to assume the server is trusted. Because of the way quake mods work, It has always been possible to have server side cheats along the lines of "if name == mine, scale damage by 75%". You have to trust the server operator.
So, the problem then becomes a matter of making sure the clients are all playing with an acceptable version before allowing them to connect to the server. You obviously can't just ask the client, because if it is hacked it can just tell you what you want to hear. Because of the nature of the GPL, you can't just have a hidden part of the code to do verification.
What needs to be done is to create two closed source programs that act as executable loaders / verifiers and communication proxies for the client and server. These would need to be produced for each platform the game runs on. Some modifications will need to be done to the open source code to allow it to (optionally) communicate with these proxies.
These programs would perform a robust binary digest of the programs they are loading and communicate with their peer in a complex encrypted protocol before allowing the game connection to start. It may be possible to bypass the proxy for normal packets to avoid adding any scheduling or latency issues, but it will need to be involved to some degree to prevent a cheater from hijacking the connection once it is created.
The server operator would determine which versions of the game are to be allowed to connect to their server if they wish to enforce proxy protection. The part of the community that wants to be competetive will have to agree to some reasonable schedule of adoption of new versions.
Nothing in online games is cheat-proof (there is allways the device driver level of things to hack on), but that would actually be more secure than the game as it originally shipped, because hex edited patches wouldn't work any more. Someone could still in theory hack the closed source programs, but that is the same situation everyone was in with the original game.
People can start working on this immediately. There is some prior art in various unix games that would probably be helpfull. It would also be a good idea to find some crypto hackers to review proposed proxy communication strategies.
-----------------------------------------
John Carmack's .plan for Dec 29, 1999
-----------------------------------------
I have been playing a lot of Q3 on a 28.8 modem for the last several days.
I finally found a case of the stuck-at-awaiting-gamestate problem that turned out to be a continuous case of a fragment of the gamestate getting dropped. I have changed the net code to space out the sending of the fragments based on rate.
Note that there have been a few different things that result in stuck at gamestate or stuck at snapshot problems. We have fixed a few of them, but there may well still be other things that we haven't found yet.
You can still have a fun game on a 28.8 modem. It is a significant disadvantage, no question about it, but you can still have a good game if you play smart. If there is someone that knows what they are doing on a server with a ping in the low 100s, there won't usually be much you can do, but a skilled modem player can still beat up on unskilled T1 players..
Make sure your modem rate is set correctly. If you have it set too high, large amounts of data can get buffered up and you can wind up with multiple seconds of screwed up delays.
Only play on servers with good pings. My connection gives me a couple dozen servers with mid 200 pings. 56k modems often see servers with sub 200 pings. If you ignore the ping and just look for your favorite map, you will probably have a crappy game.
If you have a good basic connection to the server, the thing that will mess up your game is too much visible activity. This is a characteristic of the number of players, the openness of the level, and the weapons in use.
Don't play on madhouse levels with tons of players. None of the normal Q3 maps were really designed for more than eight players, and many were only designed for four.
Don't play in the wide open maps unless there are only a couple other players. Four very active players in a wide open area are enough to bog down a modem connection.
I just implemented "sv_minPing" / "sv_maxPing" options so servers can restrict themselves to only low ping or high ping players. This is done based on the ping of the challenge response packet, rather than any in-game pings. There are a few issues with that - a LPB may occasionally get into a HPB server if they happen to get a network hiccup at just the right time, and the number used as a gate will be closer to the number shown in the server list, rather than the number seen in gameplay. I would reccomend "sv_minPing 200" as a reasonable breakpoint.
-----------------------------------------
John Carmack's .plan for Dec 30, 1999
-----------------------------------------
Several people have mentioned an existing anti-cheat QW proxy that should also be applicable to modified versions:
http://www.students.tut.fi/~zibbo/qizmo/
-----------------------------------------
John Carmack's .plan for Feb 23, 2000
-----------------------------------------
This is a public statement that is also being sent directly to Slade at QuakeLives regarding http://www.quakelives.com/main/ql.cgi?section=dlagreement&file=qwcl-win32/
I see both sides of this. Your goals are positive, and I understand the issues and the difficulties that your project has to work under because of the GPL. I have also seen some GPL zealots acting petty and immature towards you very early on (while it is within everyone's rights to DEMAND code under the GPL, it isn't necessarily the best attitude to take), which probably colors some of your views on the subject.
We discussed several possible legal solutions to the issues.
This isn't one of them.
While I doubt your "give up your rights" click through would hold up in court, I am positive that you are required to give the source to anyone that asks for it that got a binary from someone else. This doesn't provide the obscurity needed for a gaming level of security.
I cut you a lot of slack because I honestly thought you intended to properly follow through with the requirements of the GPL, and you were just trying to get something fun out ASAP. It looks like I was wrong.
If you can't stand to work under the GPL, you should release the code to your last binary and give up your project. I would prefer that you continue your work, but abide by the GPL.
If necessary, I will pay whatever lawyer the Free Software Foundation reccomends to pursue this.
-----------------------------------------
John Carmack's .plan for Feb 24, 2000
-----------------------------------------
Some people took it upon themselves to remotely wreck Slade's development system. That is no more defensible than breaking into Id and smashing something.
The idea isn't to punish anyone, it is to have them comply with the license and continue to contribute. QuakeLives has quite a few happy users, and it is in everyone's best interest to have development continue. It just has to be by the rules.
-----------------------------------------
John Carmack's .plan for Mar 07, 2000
-----------------------------------------
Virtualized video card local memory is The Right Thing.
This is something I have been preaching for a couple years, but I finally got around to setting all the issues down in writing.
Now, the argument (and a whole bunch of tertiary information):
If you had all the texture density in the world, how much texture memory would be needed on each frame?
For directly viewed textures, mip mapping keeps the amount of referenced texels between one and one quarter of the drawn pixels. When anisotropic viewing angles and upper level clamping are taken into account, the number gets smaller. Take 1/3 as a conservative estimate.
Given a fairly aggressive six texture passes over the entire screen, that equates to needing twice as many texels as pixels. At 1024x768 resolution, well under two million texels will be referenced, no matter what the finest level of detail is. This is the worst case, assuming completely unique texturing with no repeating. More commonly, less than one million texels are actually needed.
As anyone who has tried to run certain Quake 3 levels in high quality texture mode on an eight or sixteen meg card knows, it doesn't work out that way in practice. There is a fixable part and some more fundamental parts to the fall-over-dead-with-too-many-textures problem.
The fixable part is that almost all drivers perform pure LRU (least recently used) memory management. This works correctly as long as the total amount of textures needed for a given frame fits in the card's memory after they have been loaded. As soon as you need a tiny bit more memory than fits on the card, you fall off of a performance cliff. If you need 14 megs of textures to render a frame, and your graphics card has 12 megs available after its frame buffers, you wind up loading 14 megs of texture data over the bus every frame, instead of just the 2 megs that don't fit. Having the cpu generate 14 megs of command traffic can drop you way into the single digit frame rates on most drivers.
If an application makes reasonable effort to group rendering by texture, and there is some degree of coherence in the order of texture references between frames, much better performance can be gotten with a swapping algorithm that changes its behavior instead of going into a full thrash:
[code]
While ( memory allocation for new texture fails )
Find the least recently used texture.
If the LRU texture was not needed in the previous frame,
Free it
Else
Free the most recently used texture that isn't bound to an active texture unit
[/code]
Freeing the MRU texture seems counterintuitive, but what it does is cause the driver to use the last bit of memory as a sort of scratchpad that gets constantly overwritten when there isn't enough space. Pure LRU plows over all the other textures that are very likely going to be needed at the beginning of the next frame, which will then plow over all the textures that were loaded on top of them.
If an application uses textures in a completely random order, any given replacement policy has the some effect..
Texture priority for swapping is a non-feature. There is NO benefit to attempting to statically prioritize textures for swapping. Either a texture is going to be referenced in the next frame, or it isn't. There aren't any useful gradations in between. The only hint that would be useful would be a notice that a given texture is not going to be in the next frame, and that just doesn't come up very often or cover very many texels.
With the MRU-on-thrash texture swapping policy, things degrade gracefully as the total amount of textures increase but due to several issues, the total amount of textures calculated and swapped is far larger than the actual amount of texels referenced to draw pixels.
The primary problem is that textures are loaded as a complete unit, from the smallest mip map level all the way up to potentially a 2048 by 2048 top level image. Even if you are only seeing 16 pixels of it off in the distance, the entire 12 meg stack might need to be loaded.
Packing can also cause some amount of wasted texture memory. When you want to load a two meg texture, it is likely going to require a lot more than just two megs of free texture memory, because a lot of it is going to be scattered around in 8k to 64k blocks. At the pathological limit, this can waste half your texture memory, but more reasonably it is only going to be 10% or so, and cause a few extra texture swap outs.
On a frame at a time basis, there are often significant amounts of texels even in referenced mip levels that are not seen. The back sides of characters, and large textures on floors can often have less than 50% of their texels used during a frame. This is only an issue as they are being swapped in, because they will very likely be needed within the next few frames. The result is one big hitch instead of a steady loading.
There are schemes that can help with these problems, but they have costs.
Packing losses can be addressed with compaction, but that has rarely proven to be worthwhile in the history of memory management. A 128-bit graphics accelerator could compact and sort 10 megs of texture memory in about 10 msec if desired.
The problems with large textures can be solved by just not using large textures. Both packing losses, and non- referenced texels can be reduced by chopping everything up into 64x64 or 128x128 textures. This requires preprocessing, adds geometry, and requires messy overlap of the textures to avoid seaming problems.
It is possible to estimate which mip levels will actually be needed and only swap those in. An application can't calculate exactly the mip map levels that will be referenced by the hardware, because there are slight variations between chips and the slope calculation would add significant processing overhead. A conservative upper bound can be taken by looking at the minimum normal distance of any vertex referencing a given texture in a frame. This will overestimate the required textures by 2x or so and still leave a big hit when the top mip level loads for big textures, but it can allow giant cathedral style scenes to render without swapping.
Clever programmers can always work harder to overcome obstacles, but in this case, there is a clear hardware solution that gives better performance than anything possible with software and just makes everyone's lives easier: virtualize the card's view of its local memory.
With page tables, address fragmentation isn't an issue, and with the graphics rasterizer only causing a page load when something from that exact 4k block is needed, the mip level problems and hidden texture problems just go away. Nothing sneaky has to be done by the application or driver, you just manage page indexes.
The hardware requirements are not very heavy. You need translation lookaside buffers (TLB) on the graphics chip, the ability to automatically load the TLB from a page table set up in local memory, and the ability to move a page from AGP or PCI into graphics memory and update the page tables and reference counts. You don't even need that many TLB, because graphics access patterns don't hop all over the place like CPU access can. Even with only a single TLB for each texture bilerp unit, reloads would only account for about 1/32 of the memory access if the textures were 4k blocked. All you would really want at the upper limit would be enough TLB for each texture unit to cover the texels referenced on a typical rasterization scan line.
Some programmers will say "I don't want the system to manage the textures, I want full control!" There are a couple responses to that. First, a page level management scheme has flexibility that you just can't get with a software only scheme, so it is a set of brand new capabilities. Second, you can still just choose to treat it as a fixed size texture buffer and manage everything yourself with updates. Third, even if it WAS slower than the craftiest possible software scheme (and I seriously doubt it), so much of development is about willingly trading theoretical efficiency for quicker, more robust development. We don't code overlays in assembly language any more..
Some hardware designers will say something along the lines of "But the graphics engine goes idle when you are pulling the page over from AGP!" Sure, you are always better off to just have enough texture memory and never swap, and this feature wouldn't let you claim any more megapixels or megatris, but every card winds up not having enough memory at some point. Ignoring those real world cases isn't helping your customers. In any case, it goes idle a hell of a lot less than if you were loading the entire texture over the command fifo.
3Dlabs is supposed to have some form of virtual memory management in the permedia 3, but I am not familiar with the details (if anyone from 3dlabs wants to send me the latest register specs, I would appreciate it!).
A mouse controlled first person shooter is fairly unique in how quickly it can change the texture composition of a scene. A 180-degree snap turn can conceivably bring in a completely different set of textures on a subsequent frame. Almost all other graphics applications bring textures in at a much steadier pace.
So, given that 180-degree snap turn to a completely different and uniquely textured scene, what would be the worst case performance? An AGP 2x bus is theoretically supposed to have over 500 mb/sec of bandwidth. It doesn't get that high in practice, but linear 4k block reads would give it the best possible conditions, and even at 300 mb/sec, reloading the entire texture working set would only take 10 msec.
Rendering is not likely to be buffered sufficiently to overlap appreciably with page loading, and the command transport for a complex scene will take significant time by itself, so it shows that a worst case scene will often not be able to be rendered in 1/60th of a second.
This is roughly the same lower bound that you get from a chip texturing directly from AGP memory. A direct AGP texture gains the benefit of fine-grained rendering overlap, but loses the benefit of subsequent references being in faster memory (outside of small on-chip caches). A direct AGP texture engine doesn't have the higher upper bounds of a cached texture engine, though. It's best and worst case are similar (generally a good thing), but the cached system can bring several times more bandwidth to bear when it isn't forced to swap anything in.
The important point is that the lower performance bound is almost an order of magnitude faster than swapping in the textures as a unit by the driver.
If you just positively couldn't deal with the chance of that much worst case delay, some form of mip level biasing could be made to kick in, or you could try and do pre-touching, but I don't think it would ever be worth it. The worst imaginable case is acceptable, and you just won't hit that case very often.
Unless a truly large number of TLB are provided, the textures would need to be blocked. The reason is that with a linear texture, a 4k page maps to only a couple scan lines on very large textures. If you are going with the grain you get great reuse, but if you go across it, you wind up referencing a new page every couple texel accesses. What is wanted is an addressing mechanism that converts a 4k page into a square area in the texture, so the page access is roughly constant for all orientations. There is also a benefit from having a 128 bit access also map to a square block of pixels, which several existing cards already do. The same interleaving-of-low-order-bits approach can just be extended a few more bits.
Dealing with blocked texture patterns is a hassle for a driver writer, but most graphics chips have a host blit capability that should let the chip deal with changing a linear blit into blocked writes. Application developers should never know about it, in any case.
There are some other interesting things that could be done if the page tables could trigger a cpu interrupt in addition to being automatically backed by AGP or PCI memory. Textures could be paged in directly from disk for truly huge settings, or decompressed from jpeg blocks, or even procedurally generated. Even the size limits of the AGP aperture could usefully be avoided if the driver wanted to manage each page's allocation.
Aside from all the basic swapping issue, there are a couple of other hardware trends that push things this way.
Embedded dram should be a driving force. It is possible to put several megs of extremely high bandwidth dram on a chip or die with a video controller, but won't be possible (for a while) to cram a 64 meg geforce in. With virtualized texturing, the major pressure on memory is drastically reduced. Even an 8mb card would be sufficient for 16 bit 1024x768 or 32 bit 800x600 gaming, no matter what the texture load.
The only thing that prevents a geometry processor based card from turning almost any set of commands in a display list into a single static dma buffer is the fact that textures may be swapped in and out, causing the register programming in the buffer to be wrong. With virtual texture addressing, a texture's address never changes, and an arbitrarily complex model can be described in a static dma buffer.
-----------------------------------------
John Carmack's .plan for Mar 27, 2000
-----------------------------------------
Seumas McNally
Two years ago, Id was contacted by the Startlight Foundation, an organization that tries to grant wishes to seriously ill kids. (www.starlight.org)
There was a young man with Hodgkin's Lymphoma that, instead of wanting to go to Disneyland or other traditional wishes, wanted to visit Id and talk with me about programming.
It turned out that Seumas McNally was already an accomplished developer. His family company, Longbow Digital Arts (www.longbowdigitalarts.com), had been doing quite respectably selling small games directly over the internet. It bore a strong resemblance to the early shareware days of Apogee and Id.
We spent the evening talking about graphics programmer things - the relative merits of voxels and triangles, procedurally generated media, level of detail management, API and platforms.
We talked at length about the balance between technology and design, and all the pitfalls that lie in the way of shipping a modern product.
We also took a dash out in my ferrari, thinking "this is going to be the best excuse a cop will ever hear if we get pulled over".
Longbow continued to be successful, and eventually the entire family was working full time on "Treadmarks", their new 3D tank game.
Over email about finishing the technology in Treadmarks, Seumas once said "I hope I can make it". Not "be a huge success" or "beat the competition". Just "make it".
That is a yardstick to measure oneself by.
It is all too easy to lose your focus or give up with just the ordinary distractions and disappointments that life brings. This wasn't ordinary. Seumas had cancer. Whatever problems you may be dealing with in your life, they pale before having problems drawing your next breath.
He made it.
Treadmarks started shipping a couple months ago, and was entered in the Independent Games Festival at the Game Developer's Conference this last month. It came away with the awards for technical excellence, game design, and the grand prize.
I went out to dinner with the McNally family the next day, and had the opportunity to introduce Anna to them. One of the projects at Anna's new company, Fountainhead Entertainment (www.fountainheadent.com), is a documentary covering gaming, and she had been looking forward to meeting Seumas after hearing me tell his story a few times. The McNallys invited her to bring a film crew up to Canada and talk with everyone whenever she could.
Seumas died the next week.
I am proud to have been considered an influence in Seumas' work, and I think his story should be a good example for others. Through talent and determination, he took something he loved and made a success out of it in many dimensions.
http://www.gamedev.net/community/memorial/seumas/ for more information.
-----------------------------------------
John Carmack's .plan for Apr 06, 2000
-----------------------------------------
Whenever I start a new graphics engine, I always spend a fair amount of time flipping back through older graphics books. It is always interesting to see how your changed perspective with new experience impacts your appreciation of a given article.
I was skimming through Jim Blinn's "A Trip Down The Graphics Pipeline" tonight, and I wound up laughing out loud twice.
From the book:
P73: I then empirically found that I had to scale by -1 in x instead of in z, and also to scale the xa and xf values by -1. (Basically I just put in enough minus signs after the fact to make it work.) Al Barr refers to this technique as "making sure you have made an even number of sign errors."
P131: The only lines that generate w=0 after clipping are those that pass through the z axis, the valley of the trough. These lines are lines that pass exactly through the eyepoint. After which you are dead and don't care about divide-by-zero errors.
If you laughed, you are a graphics geek.
My first recollection of a Jim Blinn article many years ago was my skimming over it and thinking "My god, what ridiculously picky minutia." Over the last couple years, I found myself haranguing people over some fairly picky issues, like the LSB errors with cpu vs rasterizer face culling and screen edge clipping with guard band bit tests. After one of those pitches, I quite distinctly thought to myself "My god, I'm turning into Jim Blinn!" :)
-----------------------------------------
John Carmack's .plan for Apr 29, 2000
-----------------------------------------
We need more bits per color component in our 3D accelerators.
I have been pushing for a couple more bits of range for several years now, but I now extend that to wanting full 16 bit floating point colors throughout the graphics pipeline. A sign bit, ten bits of mantissa, and five bits of exponent (possibly trading a bit or two between the mantissa and exponent). Even that isn't all you could want, but it is the rational step.
It is turning out that I need a destination alpha channel for a lot of the new rendering algorithms, so intermediate solutions like 10/12/10 RGB formats aren't a good idea. Higher internal precision with dithering to 32 bit pixels would have some benefit, but dithered intermediate results can easily start piling up the errors when passed over many times, as we have seen with 5/6/5 rendering.
Eight bits of precision isn't enough even for full range static image display. Images with a wide range usually come out fine, but restricted range images can easily show banding on a 24-bit display. Digital television specifies 10 bits of precision, and many printing operations are performed with 12 bits of precision.
The situation becomes much worse when you consider the losses after multiple operations. As a trivial case, consider having multiple lights on a wall, with their contribution to a pixel determined by a texture lookup. A single light will fall off towards 0 some distance away, and if it covers a large area, it will have visible bands as the light adds one unit, two units, etc. Each additional light from the same relative distance stacks its contribution on top of the earlier ones, which magnifies the amount of the step between bands: instead of going 0,1,2, it goes 0,2,4, etc. Pile a few lights up like this and look towards the dimmer area of the falloff, and you can believe you are back in 256-color land.
There are other more subtle issues, like the loss of potential result values from repeated squarings of input values, and clamping issues when you sum up multiple incident lights before modulating down by a material.
Range is even more clear cut. There are some values that have intrinsic ranges of 0.0 to 1.0, like factors of reflection and filtering. Normalized vectors have a range of -1.0 to 1.0. However, the most central quantity in rendering, light, is completely unbounded. We want a LOT more than a 0.0 to 1.0 range. Q3 hacks the gamma tables to sacrifice a bit of precision to get a 0.0 to 2.0 range, but I wanted more than that for even primitive rendering techniques. To accurately model the full human sensable range of light values, you would need more than even a five bit exponent.
This wasn't much of an issue even a year ago, when we were happy to just cover the screen a couple times at a high framerate, but realtime graphics is moving away from just "putting up wallpaper" to calculating complex illumination equations at each pixel. It is not at all unreasonable to consider having twenty textures contribute to the final value of a pixel. Range and precision matter.
A few common responses to this pitch:
"64 bits per pixel??? Are you crazy???" Remember, it is exactly the same relative step as we made from 16 bit to 32 bit, which didn't take all that long.
Yes, it will be slower. That's ok. This is an important point: we can't continue to usefully use vastly greater fill rate without an increase in precision. You can always crank the resolution and multisample anti-alaising up higher, but that starts to have diminishing returns well before you use of the couple gigatexels of fill rate we are expected to have next year. The cool and interesting things to do with all that fill rate involves many passes composited into less pixels, making precision important.
"Can we just put it in the texture combiners and leave the framebuffer at 32 bits?" No. There are always going to be shade trees that overflow a given number of texture units, and they are going to be the ones that need the extra precision. Scales and biases between the framebuffer and the higher precision internal calculations can get you some mileage (assuming you can bring the blend color into your combiners, which current cards can't), but its still not what you want. There are also passes which fundamentally aren't part of a single surface, but still combine to the same pixels, as with all forms of translucency, and many atmospheric effects.
"Do we need it in textures as well?" Not for most image textures, but it still needs to be supported for textures that are used as function look up tables.
"Do we need it in the front buffer?" Probably not. Going to a 64 bit front buffer would probably play hell with all sorts of other parts of the system. It is probably reasonable to stay with 32 bit front buffers with a blit from the 64 bit back buffer performing a lookup or scale and bias operation before dithering down to 32 bit. Dynamic light adaptation can also be done during this copy. Dithering can work quite well as long as you are only performing a single pass.
I used to be pitching this in an abstract "you probably should be doing this" form, but two significant things have happened that have moved this up my hit list to something that I am fairly positive about.
Mark Peercy of SGI has shown, quite surprisingly, that all Renderman surface shaders can be decomposed into multi-pass graphics operations if two extensions are provided over basic OpenGL: the existing pixel texture extension, which allows dependent texture lookups (matrox already supports a form of this, and most vendors will over the next year), and signed, floating point colors through the graphics pipeline. It also makes heavy use of the existing, but rarely optimized, copyTexSubImage2D functionality for temporaries.
This is a truly striking result. In retrospect, it seems obvious that with adds, multiplies, table lookups, and stencil tests that you can perform any computation, but most people were working under the assumption that there were fundamentally different limitations for "realtime" renderers vs offline renderers. It may take hundreds or thousands of passes, but it clearly defines an approach with no fundamental limits. This is very important. I am looking forward to his Siggraph paper this year.
Once I set down and started writing new renderers targeted at GeForce level performance, the precision issue has started to bite me personally. There are quite a few times where I have gotten visible banding after a set of passes, or have had to worry about ordering operations to avoid clamping. There is nothing like actually dealing with problems that were mostly theoretical before..
64 bit pixels. It is The Right Thing to do. Hardware vendors: don't you be the company that is the last to make the transition.
-----------------------------------------
John Carmack's .plan for May 08, 2000
-----------------------------------------
The .qc files for quake1/quakeworld are now available under the GPL in source/qw-qc.tar.gx on out ftp site. This was an oversight on my part in the original release.
Thanks to the QuakeForge team for doing the grunt work of the preparation.
-----------------------------------------
John Carmack's .plan for May 09, 2000
-----------------------------------------
And the Q1 utilities are now also available under the GPL in source/q1tools_gpl.tgz.
-----------------------------------------
John Carmack's .plan for May 14, 2000
-----------------------------------------
I stayed a couple days after E3 to attend the SORAC amateur rocket launch. I have provided some sponsorship to two of the teams competing for the CATS (Cheap Access to Space) rocketry prize, and it was a nice opportunity to get out and meet some of the people.
It is interesting how similar the activity is around an experimental rocket launch, going to a race track with an experimental car, and putting out a beta version of new software is. Lots of "twenty more minutes!", and lots of well-wishers waiting around while the people on the critical path sweat over what they are doing.
Mere minutes before we absolutely, positively needed to leave to catch our plane flight, they started the countdown. The rocket launched impressively, but broke apart at a relatively low altitude. Ouch. It was a hybrid, so there wasn't really an explosion, but watching the debris rain down wasn't very heartening. Times like that, I definitely appreciate working in software. "Run it again, with a breakpoint!"
Note to self: pasty-skinned programmers ought not stand out in the Mojave desert for multiple hours.
http://www.space-frontier.org/Events/CATSPRIZE_1/
http://www.energyrs.com/sorac/sorac.htm
http://www.jpaerospace.com/
-----------------------------------------
John Carmack's .plan for May 17, 2000
-----------------------------------------
I have gotten a lot of requests for comments on the latest crop of video cards, so here is my initial technical evaluation. We have played with some early versions, but this is a paper evaluation. I am not in a position to judge 2D GDI issues or TV/DVD issues, so this is just 3D commentary.
Nvidia Marketing silliness: saying "seven operations on a pixel" for a dual texture chip. Yes, I like NV_register_combiners a lot, but come on..
The DDR GeForce is the reigning champ of 3D cards. Of the shipping boards, it is basically better than everyone at every aspect of 3D graphics, and pioneered some features that are going to be very important: signed pixel math, dot product blending, and cubic environment maps.
The GeForce2 is just a speed bumped GeForce with a few tweaks, but that's not a bad thing. Nvidia will have far and away the tightest drivers for quite some time, and that often means more than a lot of new features in the real world.
The nvidia register combiners are highly programmable, and can often save a rendering pass or allow a somewhat higher quality calculation, but on the whole, I would take ATI's third texture for flexibility.
Nvidia will probably continue to hit the best framerates in benchmarks at low resolution, because they have flexible hardware with geometry acceleration and well-tuned drivers.
GeForce is my baseline for current rendering work, so I can wholeheartedly recommend it.
ATI Marketing silliness: "charisma engine" and "pixel tapestry" are silly names for vertex and pixel processing that are straightforward improvements over existing methods. Sony is probably to blame for starting that.
The Radeon has the best feature set available, with several advantages over GeForce:
A third texture unit per pixel
Three dimensional textures
Dependent texture reads (bump env map)
Greater internal color precision.
User clip planes orthogonal to all rasterization modes.
More powerful vertex blending operations.
The shadow id map support may be useful, but my work with shadow buffers have shown them to have significant limitations for global use in a game.
On paper, it is better than GeForce in almost every way except that it is limited to a maximum of two pixels per clock while GeForce can do four. This comes into play when the pixels don't do as much memory access, for example when just drawing shadow planes to the depth/stencil buffer, or when drawing in roughly front to back order and many of the later pixels depth fail, avoiding the color buffer writes.
Depending on the application and algorithm, this can be anywhere from basically no benefit when doing 32 bit blended multi-pass, dual texture rendering to nearly double the performance for 16 bit rendering with compressed textures. In any case, a similarly clocked GeForce(2) should somewhat outperform a Radeon on today's games when fill rate limited. Future games that do a significant number of rendering passes on the entire world may go back in ATI's favor if they can use the third texture unit, but I doubt it will be all that common.
The real issue is how quickly ATI can deliver fully clocked production boards, bring up stable drivers, and wring all the performance out of the hardware. This is a very different beast than the Rage128. I would definitely recommend waiting on some consumer reviews to check for teething problems before upgrading to a Radeon, but if things go well, ATI may give nvidia a serious run for their money this year.
3DFX Marketing silliness: Implying that a voodoo 5 is of a different class than a voodoo 4 isn't right. Voodoo 4 max / ultra / SLI / dual / quad or something would have been more forthright.
Rasterization feature wise, voodoo4 is just catching up to the original TNT. We finally have 32 bit color and stencil. Yeah.
There aren't any geometry features.
The T buffer is really nothing more than an accumulation buffer that is averaged together during video scanout. This same combining of separate buffers can be done by any modern graphics card if they are set up for it (although they will lose two bits of color precision in the process). At around 60 fps there is a slight performance win by doing it at video scannout time, but at 30 fps it is actually less memory traffic to do it explicitly. Video scan tricks also usually don't work in windowed modes.
The real unique feature of the voodoo5 is subpixel jittering during rasterization, which can't reasonably be emulated by other hardware. This does indeed improve the quality of anti-aliasing, although I think 3dfx might be pushing it a bit by saying their 4 sample jittering is as good as 16 sample unjittered.
The saving grace of the voodoo5 is the scalability. Because it only uses SDR ram, a dual chip Voodoo5 isn't all that much faster than some other single chip cards, but the quad chip card has over twice the pixel fill rate of the nearest competitor. That is a huge increment. Voodoo5 6000 should win every benchmark that becomes fill rate limited.
I haven't been able to honestly recommend a voodoo3 to people for a long time, unless they had a favorite glide game or wanted early linux Xfree 4.0 3D support. Now (well, soon), a Voodoo5 6000 should make all of today's games look better than any other card. You can get over twice as many pixel samples, and have them jittered and blended together for anti-aliasing.
It won't be able to hit Q3 frame rates as high as GeForce, but if you have a high end processor there really may not be all that much difference for you between 100fps and 80fps unless you are playing hardcore competitive and can't stand the occasional drop below 60fps.
There are two drawbacks: it's expensive, and it won't take advantage of the new rasterization features coming in future games. It probably wouldn't be wise to buy a voodoo5 if you plan on keeping it for two years.
-----------------------------------------
John Carmack's .plan for Jun 01, 2000
-----------------------------------------
Well, this is going to be an interesting .plan update.
Most of this is not really public business, but if some things aren't stated explicitly, it will reflect unfairly on someone.
As many people have heard discussed, there was quite a desire to remake DOOM as our next project after Q3. Discussing it brought an almost palpable thrill to most of the employees, but Adrian had a strong enough dislike for the idea that it was shot down over and over again.
Design work on an alternate game has been going on in parallel with the mission pack development and my research work.
Several factors, including a general lack of enthusiasm for the proposed plan, the warmth that Wolfenstien was met with at E3, and excitement about what we can do with the latest rendering technology were making it seem more and more like we weren't going down the right path.
I discussed it with some of the other guys, and we decided that it was important enough to drag the company through an unpleasant fight over it.
An ultimatum was issued to Kevin and Adrian(who control >50% of the company): We are working on DOOM for the next project unless you fire us.
Obviously no fun for anyone involved, but the project direction was changed, new hires have been expedited, and the design work has begun.
It wasn't planned to announce this soon, but here it is: We are working on a new DOOM game, focusing on the single player game experience, and using brand new technology in almost every aspect of it. That is all we are prepared to say about the game for quite some time, so don't push for interviews. We will talk about it when things are actually built, to avoid giving misleading comments.
It went smoother than expected, but the other shoe dropped yesterday.
Kevin and Adrian fired Paul Steed in retaliation, over my opposition.
Paul has certainly done things in the past that could be grounds for dismissal, but this was retaliatory for him being among the "conspirators".
I happen to think Paul was damn good at his job, and that he was going to be one of the most valuable contributors to DOOM.
We need to hire two new modeler/animator/cinematic director types. If you have a significant commercial track record in all three areas, and consider yourself at the top of your field, send your resume to Kevin Cloud.
-----------------------------------------
John Carmack's .plan for Feb 22, 2001
-----------------------------------------
GeForce 3 Overview
I just got back from Tokyo, where I demonstrated our new engine running under MacOS-X with a GeForce 3 card. We had quite a bit of discussion about whether we should be showing anything at all, considering how far away we are from having a title on the shelves, so we probably aren't going to be showing it anywhere else for quite a while.
We do run a bit better on a high end wintel system, but the Apple performance is still quite good, especially considering the short amount of time that the drivers had before the event.
It is still our intention to have a simultaneous release of the next product on Windows, MacOS-X, and Linux.
Here is a dump on the GeForce 3 that I have been seriously working with for a few weeks now:
The short answer is that the GeForce 3 is fantastic. I haven't had such an impression of raising the performance bar since the Voodoo 2 came out, and there are a ton of new features for programmers to play with.
Graphics programmers should run out and get one at the earliest possible time. For consumers, it will be a tougher call. There aren't any applications our right now that take proper advantage of it, but you should still be quite a bit faster at everything than GF2, especially with anti-aliasing. Balance that against whatever the price turns out to be.
While the Radeon is a good effort in many ways, it has enough shortfalls that I still generally call the GeForce 2 ultra the best card you can buy right now, so Nvidia is basically dethroning their own product.
It is somewhat unfortunate that it is labeled GeForce 3, because GeForce 2 was just a speed bump of GeForce, while GF3 is a major architectural change. I wish they had called the GF2 something else.
The things that are good about it:
Lots of values have additional internal precision, like texture coordinates and rasterization coordinates. There are only a few places where this matters, but it is nice to be cleaning up. Rasterization precision is about the last thing that the multi-thousand dollar workstation boards still do any better than the consumer cards.
Adding more texture units and more register combiners is an obvious evolutionary step.
An interesting technical aside: when I first changed something I was doing with five single or dual texture passes on a GF to something that only took two quad texture passes on a GF3, I got a surprisingly modest speedup. It turned out that the texture filtering and bandwidth was the dominant factor, not the frame buffer traffic that was saved with more texture units. When I turned off anisotropic filtering and used compressed textures, the GF3 version became twice as fast.
The 8x anisotropic filtering looks really nice, but it has a 30%+ speed cost. For existing games where you have speed to burn, it is probably a nice thing to force on, but it is a bit much for me to enable on the current project. Radeon supports 16x aniso at a smaller speed cost, but not in conjunction with trilinear, and something is broken in the chip that makes the filtering jump around with triangular rasterization dependencies.
The depth buffer optimizations are similar to what the Radeon provides, giving almost everything some measure of speedup, and larger ones available in some cases with some redesign.
3D textures are implemented with the full, complete generality. Radeon offers 3D textures, but without mip mapping and in a non-orthogonal manner (taking up two texture units).
Vertex programs are probably the most radical new feature, and, unlike most "radical new features", actually turn out to be pretty damn good. The instruction language is clear and obvious, with wonderful features like free arbitrary swizzle and negate on each operand, and the obvious things you want for graphics like dot product instructions.
The vertex program instructions are what SSE should have been.
A complex setup for a four-texture rendering pass is way easier to understand with a vertex program than with a ton of texgen/texture matrix calls, and it lets you do things that you just couldn't do hardware accelerated at all before. Changing the model from fixed function data like normals, colors, and texcoords to generalized attributes is very important for future progress.
Here, I think Microsoft and DX8 are providing a very good benefit by forcing a single vertex program interface down all the hardware vendor's throats.
This one is truly stunning: the drivers just worked for all the new features that I tried. I have tested a lot of pre-production 3D cards, and it has never been this smooth.
The things that are indifferent:
I'm still not a big believer in hardware accelerated curve tessellation. I'm not going to go over all the reasons again, but I would have rather seen the features left off and ended up with a cheaper part.
The shadow map support is good to get in, but I am still unconvinced that a fully general engine can be produced with acceptable quality using shadow maps for point lights. I spent a while working with shadow buffers last year, and I couldn't get satisfactory results. I will revisit that work now that I have GeForce 3 cards, and directly compare it with my current approach.
At high triangle rates, the index bandwidth can get to be a significant thing. Other cards that allow static index buffers as well as static vertex buffers will have situations where they provide higher application speed. Still, we do get great throughput on the GF3 using vertex array range and glDrawElements.
The things that are bad about it:
Vertex programs aren't invariant with the fixed function geometry paths. That means that you can't mix vertex program passes with normal passes in a multipass algorithm. This is annoying, and shouldn't have happened.
Now we come to the pixel shaders, where I have the most serious issues. I can just ignore this most of the time, but the way the pixel shader functionality turned out is painfully limited, and not what it should have been.
DX8 tries to pretend that pixel shaders live on hardware that is a lot more general than the reality.
Nvidia's OpenGL extensions expose things much more the way they actually are: the existing register combiners functionality extended to eight stages with a couple tweaks, and the texture lookup engine is configurable to interact between textures in a list of specific ways.
I'm sure it started out as a better design, but it apparently got cut and cut until it really looks like the old BumpEnvMap feature writ large: it does a few specific special effects that were deemed important, at the expense of a properly general solution.
Yes, it does full bumpy cubic environment mapping, but you still can't just do some math ops and look the result up in a texture. I was disappointed on this count with the Radeon as well, which was just slightly too hardwired to the DX BumpEnvMap capabilities to allow more general dependent texture use.
Enshrining the capabilities of this mess in DX8 sucks. Other companies had potentially better approaches, but they are now forced to dumb them down to the level of the GF3 for the sake of compatibility. Hopefully we can still see some of the extra flexibility in OpenGL extensions.
The future:
I think things are going to really clean up in the next couple years. All of my advocacy is focused on making sure that there will be a completely clean and flexible interface for me to target in the engine after DOOM, and I think it is going to happen.
The market may have shrunk to just ATI and Nvidia as significant players. Matrox, 3D labs, or one of the dormant companies may surprise us all, but the pace is pretty frantic.
I think I would be a little more comfortable if there was a third major player competing, but I can't fault Nvidia's path to success.
-----------------------------------------
John Carmack's .plan for Nov 16, 2001
-----------------------------------------
Driver optimizations have been discussed a lot lately because of the quake3 name checking in ATI's recent drivers, so I am going to lay out my position on the subject.
There are many driver optimizations that are pure improvements in all cases, with no negative effects. The difficult decisions come up when it comes to "trades" of various kinds, where a change will give an increase in performance, but at a cost.
Relative performance trades. Part of being a driver writer is being able to say "I don't care if stippled, anti-aliased points with texturing go slow", and optimizing accordingly. Some hardware features, like caches and hierarchical buffers, may be advantages on some apps, and disadvantages on others. Command buffer sizes often tune differently for different applications.
Quality trades. There is a small amount of wiggle room in the specs for pixel level variability, and some performance gains can be had by leaning towards the minimums. Most quality trades would actually be conformance trades, because the results are not exactly conformant, but they still do "roughly" the right thing from a visual standpoint. Compressing textures automatically, avoiding blending of very faint transparent pixels, using a 16 bit depth buffer, etc. A good application will allow the user to make most of these choices directly, but there is good call for having driver preference panels to enable these types of changes on naive applications. Many drivers now allow you to quality trade in an opposite manner - slowing application performance by turning on anti-aliasing or anisotropic texture filtering.
Conformance trades. Most conformance trades that happen with drivers are unintentional, where the slower, more general fallback case just didn't get called when it was supposed to, because the driver didn't check for a certain combination to exit some specially optimized path. However, there are optimizations that can give performance improvements in ways that make it impossible to remain conformant. For example, a driver could choose to skip storing of a color value before it is passed on to the hardware, which would save a few cycles, but make it impossible to correctly answer glGetFloatv( GL_CURRENT_COLOR, buffer ).
Normally, driver writers will just pick their priorities and make the trades, but sometimes there will be a desire to make different trades in different circumstances, so as to get the best of both worlds.
Explicit application hints are a nice way to offer different performance characteristics, but that requires cooperation from the application, so it doesn't help in an ongoing benchmark battle. OpenGL's glHint() call is the right thought, but not really set up as flexibly as you would like. Explicit extensions are probably the right way to expose performance trades, but it isn't clear to me that any conformant trade will be a big enough difference to add code for.
End-user selectable optimizations. Put a selection option in the driver properties window to allow the user to choose which application class they would like to be favored in some way. This has been done many times, and is a reasonable way to do things. Most users would never touch the setting, so some applications may be slightly faster or slower than in their "optimal benchmark mode".
Attempt to guess the application from app names, window strings, etc. Drivers are sometimes forced to do this to work around bugs in established software, and occasionally they will try to use this as a cue for certain optimizations.
My positions:
Making any automatic optimization based on a benchmark name is wrong. It subverts the purpose of benchmarking, which is to gauge how a similar class of applications will perform on a tested configuration, not just how the single application chosen as representative performs.
It is never acceptable to have the driver automatically make a conformance tradeoff, even if they are positive that it won't make any difference. The reason is that applications evolve, and there is no guarantee that a future release won't have different assumptions, causing the upgrade to misbehave. We have seen this in practice with Quake3 and derivatives, where vendors assumed something about what may or may not be enabled during a compiled vertex array call. Most of these are just mistakes, or, occasionally, laziness.
Allowing a driver to present a non-conformant option for the user to select is an interesting question. I know that as a developer, I would get hate mail from users when a point release breaks on their whiz-bang optimized driver, just like I do with overclocked CPUs, and I would get the same "but it works with everything else!" response when I tell them to put it back to normal. On the other hand, being able to tweak around with that sort of think is fun for technically inclined users. I lean towards frowning on it, because it is a slippery slope from there down in to "cheating drivers" of the see-through- walls variety.
Quality trades are here to stay, with anti-aliasing, anisotropic texture filtering, and other options being positive trades that a user can make, and allowing various texture memory optimizations can be a very nice thing for a user trying to get some games to work well. However, it is still important that it start from a completely conformant state by default. This is one area where application naming can be used reasonably by the driver, to maintain user selected per-application modifiers.
I'm not fanatical on any of this, because the overriding purpose of software is to be useful, rather than correct, but the days of game-specific mini- drivers that can just barely cut it are past, and we should demand more from the remaining vendors.
Also, excessive optimization is the cause of quite a bit of ill user experience with computers. Byzantine code paths extract costs as long as they exist, not just as they are written.
-----------------------------------------
John Carmack's .plan for Dec 21, 2001
-----------------------------------------
The Quake 2 source code is now available for download, licensed under the GPL.
ftp://ftp.idsoftware.com/idstuff/source/quake2.zip
As with previous source code releases, the game data remains under the original copyright and license, and cannot be freely distributed. If you create a true total conversion, you can give (or sell) a complete package away, as long as you abide by the GPL source code license. If your projects use the original Quake 2 media, the media must come from a normal, purchased copy of the game.
I'm sure I will catch some flack about increased cheating after the source release, but there are plenty of Q2 cheats already out there, so you are already in the position of having to trust the other players to a degree. The problem is really only solvable by relying on the community to police itself, because it is a fundamentally unwinnable technical battle to make a completely cheat proof game of this type. Play with your friends.
-----------------------------------------
John Carmack's .plan for Feb 11, 2002
-----------------------------------------
Last month I wrote the Radeon 8500 support for Doom.
The bottom line is that it will be a fine card for the game, but the details are sort of interesting.
I had a pre-production board before Siggraph last year, and we were discussing the possibility of letting ATI show a Doom demo behind closed doors on it. We were all very busy at the time, but I took a shot at bringing up support over a weekend. I hadn't coded any of the support for the custom ATI extensions yet, but I ran the game using only standard OpenGL calls (this is not a supported path, because without bump mapping everything looks horrible) to see how it would do. It didn't even draw the console correctly, because they had driver bugs with texGen. I thought the odds were very long against having all the new, untested extensions working properly, so I pushed off working on it until they had revved the drivers a few more times.
My judgment was colored by the experience of bringing up Doom on the original Radeon card a year earlier, which involved chasing a lot of driver bugs. Note that ATI was very responsive, working closely with me on it, and we were able to get everything resolved, but I still had no expectation that things would work correctly the first time.
Nvidia's OpenGL drivers are my "gold standard", and it has been quite a while since I have had to report a problem to them, and even their brand new extensions work as documented the first time I try them. When I have a problem on an Nvidia, I assume that it is my fault. With anyone else's drivers, I assume it is their fault. This has turned out correct almost all the time. I have heard more anecdotal reports of instability on some systems with Nivida drivers recently, but I track stability separately from correctness, because it can be influenced by so many outside factors.
ATI had been patiently pestering me about support for a few months, so last month I finally took another stab at it. The standard OpenGL path worked flawlessly, so I set about taking advantage of all the 8500 specific features. As expected, I did run into more driver bugs, but ATI got me fixes rapidly, and we soon had everything working properly. It is interesting to contrast the Nvidia and ATI functionality:
The vertex program extensions provide almost the same functionality. The ATI hardware is a little bit more capable, but not in any way that I care about. The ATI extension interface is massively more painful to use than the text parsing interface from nvidia. On the plus side, the ATI vertex programs are invariant with the normal OpenGL vertex processing, which allowed me to reuse a bunch of code. The Nvidia vertex programs can't be used in multipass algorithms with standard OpenGL passes, because they generate tiny differences in depth values, forcing you to implement EVERYTHING with vertex programs. Nvidia is planning on making this optional in the future, at a slight speed cost.
I have mixed feelings about the vertex object / vertex array range extensions. ATI's extension seems more "right" in that it automatically handles synchronization by default, and could be implemented as a wire protocol, but there are advantages to the VAR extension being simply a hint. It is easy to have a VAR program just fall back to normal virtual memory by not setting the hint and using malloc, but ATI's extension requires different function calls for using vertex objects and normal vertex arrays.
The fragment level processing is clearly way better on the 8500 than on the Nvidia products, including the latest GF4. You have six individual textures, but you can access the textures twice, giving up to eleven possible texture accesses in a single pass, and the dependent texture operation is much more sensible. This wound up being a perfect fit for Doom, because the standard path could be implemented with six unique textures, but required one texture (a normalization cube map) to be accessed twice. The vast majority of Doom light / surface interaction rendering will be a single pass on the 8500, in contrast to two or three passes, depending on the number of color components in a light, for GF3/GF4 (*note GF4 bitching later on).
Initial performance testing was interesting. I set up three extreme cases to exercise different characteristics:
A test of the non-textured stencil shadow speed showed a GF3 about 20% faster than the 8500. I believe that Nvidia has a slightly higher performance memory architecture.
A test of light interaction speed initially had the 8500 significantly slower than the GF3, which was shocking due to the difference in pass count. ATI identified some driver issues, and the speed came around so that the 8500 was faster in all combinations of texture attributes, in some cases 30+% more. This was about what I expected, given the large savings in memory traffic by doing everything in a single pass.
A high polygon count scene that was more representative of real game graphics under heavy load gave a surprising result. I was expecting ATI to clobber Nvidia here due to the much lower triangle count and MUCH lower state change functional overhead from the single pass interaction rendering, but they came out slower. ATI has identified an issue that is likely causing the unexpected performance, but it may not be something that can be worked around on current hardware.
I can set up scenes and parameters where either card can win, but I think that current Nvidia cards are still a somewhat safer bet for consistent performance and quality.
On the topic of current Nvidia cards:
Do not buy a GeForce4-MX for Doom.
Nvidia has really made a mess of the naming conventions here. I always thought it was bad enough that GF2 was just a speed bumped GF1, while GF3 had significant architectural improvements over GF2. I expected GF4 to be the speed bumped GF3, but calling the NV17 GF4-MX really sucks.
GF4-MX will still run Doom properly, but it will be using the NV10 codepath with only two texture units and no vertex shaders. A GF3 or 8500 will be much better performers. The GF4-MX may still be the card of choice for many people depending on pricing, especially considering that many games won't use four textures and vertex programs, but damn, I wish they had named it something else.
As usual, there will be better cards available from both Nvidia and ATI by the time we ship the game.
8:50 pm addendum: Mark Kilgard at Nvidia said that the current drivers already support the vertex program option to be invarint with the fixed function path, and that it turned out to be one instruction FASTER, not slower.
-----------------------------------------
John Carmack's .plan for Mar 15, 2002
-----------------------------------------
Mark Kilgard and Cass Everitt at Nvidia have released a paper on shadow volume rendering with several interesting bits in it. They also include a small document that I wrote a couple years ago about my discovery process during the development of some of the early Doom technology.
http://developer.nvidia.com/view.asp?IO=robust_shadow_volumes
-----------------------------------------
John Carmack's .plan for Jun 25, 2002
-----------------------------------------
The Matrox Parhelia Report:
The executive summary is that the Parhelia will run Doom, but it is not performance competitive with Nvidia or ATI.
Driver issue remain, so it is not perfect yet, but I am confident that Matrox will resolve them.
The performance was really disappointing for the first 256 bit DDR card. I tried to set up a "poster child" case that would stress the memory subsystem above and beyond any driver or triangle level inefficiencies, but I was unable to get it to ever approach the performance of a GF4.
The basic hardware support is good, with fragment flexibility better than GF4 (but not as good as ATI 8500), but it just doesn't keep up in raw performance. With a die shrink, this chip could probably be a contender, but there are probably going to be other chips out by then that will completely eclipse this generation of products.
None of the special features will be really useful for Doom:
The 10 bit color framebuffer is nice, but Doom needs more than 2 bits of destination alpha when a card only has four texture units, so we can't use it.
Anti aliasing features are nice, but it isn't all that fast in minimum feature mode, so nobody is going to be turning on AA. The same goes for "surround gaming". While the framerate wouldn't be 1/3 the base, it would still probably be cut in half.
Displacement mapping. Sigh. I am disappointed that the industry is still pursuing any quad based approaches. Haven't we learned from the stellar success of 3DO, Saturn, and NV1 that quads really suck? In any case, we can't use any geometry amplification scheme (including ATI's truform) in conjunction with stencil shadow volumes.
-----------------------------------------
John Carmack's .plan for Jun 27, 2002
-----------------------------------------
More graphics card notes:
I need to apologize to Matrox - their implementation of hardware displacement mapping is NOT quad based. I was thinking about a certain other companies proposed approach. Matrox's implementation actually looks quite good, so even if we don't use it because of the geometry amplification issues, I think it will serve the noble purpose of killing dead any proposal to implement a quad based solution.
I got a 3Dlabs P10 card in last week, and yesterday I put it through its paces. Because my time is fairly over committed, first impressions often determine how much work I devote to a given card. I didn't speak to ATI for months after they gave me a beta 8500 board last year with drivers that rendered the console incorrectly. :)
I was duly impressed when the P10 just popped right up with full functional support for both the fallback ARB_ extension path (without specular highlights), and the NV10 NVidia register combiners path. I only saw two issues that were at all incorrect in any of our data, and one of them is debatable. They don't support NV_vertex_program_1_1, which I use for the NV20 path, and when I hacked my programs back to 1.0 support for testing, an issue did show up, but still, this is the best showing from a new board from any company other than Nvidia.
It is too early to tell what the performance is going to be like, because they don't yet support a vertex object extension, so the CPU is hand feeding all the vertex data to the card at the moment. It was faster than I expected for those circumstances.
Given the good first impression, I was willing to go ahead and write a new back end that would let the card do the entire Doom interaction rendering in a single pass. The most expedient sounding option was to just use the Nvidia extensions that they implement, NV_vertex_program and NV_register_combiners, with seven texture units instead of the four available on GF3/GF4. Instead, I decided to try using the prototype OpenGL 2.0 extensions they provide.
The implementation went very smoothly, but I did run into the limits of their current prototype compiler before the full feature set could be implemented. I like it a lot. I am really looking forward to doing research work with this programming model after the compiler matures a bit. While the shading languages are the most critical aspects, and can be broken out as extensions to current OpenGL, there are a lot of other subtle-but-important things that are addressed in the full OpenGL 2.0 proposal.
I am now committed to supporting an OpenGL 2.0 renderer for Doom through all the spec evolutions. If anything, I have been somewhat remiss in not pushing the issues as hard as I could with all the vendors. Now really is the critical time to start nailing things down, and the decisions may stay with us for ten years.
A GL2 driver won't give any theoretical advantage over the current back ends optimized for cards with 7+ texture capability, but future research work will almost certainly be moving away from the lower level coding practices, and if some new vendor pops up (say, Rendition back from the dead) with a next-gen card, I would strongly urge them to implement GL2 instead of proprietary extensions.
I have not done a detailed comparison with Cg. There are a half dozen C-like graphics languages floating around, and honestly, I don't think there is a hell of a lot of usability difference between them at the syntax level. They are all a whole lot better than the current interfaces we are using, so I hope syntax quibbles don't get too religious. It won't be too long before all real work is done in one of these, and developers that stick with the lower level interfaces will be regarded like people that write all-assembly PC applications today. (I get some amusement from the all-assembly crowd, and it can be impressive, but it is certainly not effective)
I do need to get up on a soapbox for a long discourse about why the upcoming high level languages MUST NOT have fixed, queried resource limits if they are going to reach their full potential. I will go into a lot of detail when I get a chance, but drivers must have the right and responsibility to multipass arbitrarily complex inputs to hardware with smaller limits. Get over it.
-----------------------------------------
John Carmack's .plan for Jan 29, 2003
-----------------------------------------
NV30 vs R300, current developments, etc
At the moment, the NV30 is slightly faster on most scenes in Doom than the R300, but I can still find some scenes where the R300 pulls a little bit ahead. The issue is complicated because of the different ways the cards can choose to run the game.
The R300 can run Doom in three different modes: ARB (minimum extensions, no specular highlights, no vertex programs), R200 (full featured, almost always single pass interaction rendering), ARB2 (floating point fragment shaders, minor quality improvements, always single pass).
The NV30 can run DOOM in five different modes: ARB, NV10 (full featured, five rendering passes, no vertex programs), NV20 (full featured, two or three rendering passes), NV30 ( full featured, single pass), and ARB2.
The R200 path has a slight speed advantage over the ARB2 path on the R300, but only by a small margin, so it defaults to using the ARB2 path for the quality improvements. The NV30 runs the ARB2 path MUCH slower than the NV30 path. Half the speed at the moment. This is unfortunate, because when you do an exact, apples-to-apples comparison using exactly the same API, the R300 looks twice as fast, but when you use the vendor-specific paths, the NV30 wins.
The reason for this is that ATI does everything at high precision all the time, while Nvidia internally supports three different precisions with different performances. To make it even more complicated, the exact precision that ATI uses is in between the floating point precisions offered by Nvidia, so when Nvidia runs fragment programs, they are at a higher precision than ATI's, which is some justification for the slower speed. Nvidia assures me that there is a lot of room for improving the fragment program performance with improved driver compiler technology.
The current NV30 cards do have some other disadvantages: They take up two slots, and when the cooling fan fires up they are VERY LOUD. I'm not usually one to care about fan noise, but the NV30 does annoy me.
I am using an NV30 in my primary work system now, largely so I can test more of the rendering paths on one system, and because I feel Nvidia still has somewhat better driver quality (ATI continues to improve, though). For a typical consumer, I don't think the decision is at all clear cut at the moment.
For developers doing forward looking work, there is a different tradeoff - the NV30 runs fragment programs much slower, but it has a huge maximum instruction count. I have bumped into program limits on the R300 already.
As always, better cards are coming soon.
-----
Doom has dropped support for vendor-specific vertex programs (NV_vertex_program and EXT_vertex_shader), in favor of using ARB_vertex_program for all rendering paths. This has been a pleasant thing to do, and both ATI and Nvidia supported the move. The standardization process for ARB_vertex_program was pretty drawn out and arduous, but in the end, it is a just-plain-better API than either of the vendor specific ones that it replaced. I fretted for a while over whether I should leave in support for the older APIs for broader driver compatibility, but the final decision was that we are going to require a modern driver for the game to run in the advanced modes. Older drivers can still fall back to either the ARB or NV10 paths.
The newly-ratified ARB_vertex_buffer_object extension will probably let me do the same thing for NV_vertex_array_range and ATI_vertex_array_object.
Reasonable arguments can be made for and against the OpenGL or Direct-X style of API evolution. With vendor extensions, you get immediate access to new functionality, but then there is often a period of squabbling about exact feature support from different vendors before an industry standard settles down. With central planning, you can have "phasing problems" between hardware and software releases, and there is a real danger of bad decisions hampering the entire industry, but enforced commonality does make life easier for developers. Trying to keep boneheaded-ideas-that-will-haunt-us-for-years out of Direct-X is the primary reason I have been attending the Windows Graphics Summit for the past three years, even though I still code for OpenGL.
The most significant functionality in the new crop of cards is the truly flexible fragment programming, as exposed with ARB_fragment_program. Moving from the "switches and dials" style of discrete functional graphics programming to generally flexible programming with indirection and high precision is what is going to enable the next major step in graphics engines.
It is going to require fairly deep, non-backwards-compatible modifications to an engine to take real advantage of the new features, but working with ARB_fragment_program is really a lot of fun, so I have added a few little tweaks to the current codebase on the ARB2 path:
High dynamic color ranges are supported internally, rather than with post-blending. This gives a few more bits of color precision in the final image, but it isn't something that you really notice.
Per-pixel environment mapping, rather than per-vertex. This fixes a pet-peeve of mine, which is large panes of environment mapped glass that aren't tessellated enough, giving that awful warping-around-the-triangulation effect as you move past them.
Light and view vectors normalized with math, rather than a cube map. On future hardware this will likely be a performance improvement due to the decrease in bandwidth, but current hardware has the computation and bandwidth balanced such that it is pretty much a wash. What it does (in conjunction with floating point math) give you is a perfectly smooth specular highlight, instead of the pixelish blob that we get on older generations of cards.
There are some more things I am playing around with, that will probably remain in the engine as novelties, but not supported features:
Per-pixel reflection vector calculations for specular, instead of an interpolated half-angle. The only remaining effect that has any visual dependency on the underlying geometry is the shape of the specular highlight. Ideally, you want the same final image for a surface regardless of if it is two giant triangles, or a mesh of 1024 triangles. This will not be true if any calculation done at a vertex involves anything other than linear math operations. The specular half-angle calculation involves normalizations, so the interpolation across triangles on a surface will be dependent on exactly where the vertexes are located. The most visible end result of this is that on large, flat, shiny surfaces where you expect a clean highlight circle moving across it, you wind up with a highlight that distorts into an L shape around the triangulation line.
The extra instructions to implement this did have a noticeable performance hit, and I was a little surprised to see that the highlights not only stabilized in shape, but also sharpened up quite a bit, changing the scene more than I expected. This probably isn't a good tradeoff today for a gamer, but it is nice for any kind of high-fidelity rendering.
Renormalization of surface normal map samples makes significant quality improvements in magnified textures, turning tight, blurred corners into shiny, smooth pockets, but it introduces a huge amount of aliasing on minimized textures. Blending between the cases is possible with fragment programs, but the performance overhead does start piling up, and it may require stashing some information in the normal map alpha channel that varies with mip level. Doing good filtering of a specularly lit normal map texture is a fairly interesting problem, with lots of subtle issues.
Bump mapped ambient lighting will give much better looking outdoor and well-lit scenes. This only became possible with dependent texture reads, and it requires new designer and tool-chain support to implement well, so it isn't easy to test globally with the current Doom datasets, but isolated demos are promising.
The future is in floating point framebuffers. One of the most noticeable thing this will get you without fundamental algorithm changes is the ability to use a correct display gamma ramp without destroying the dark color precision. Unfortunately, using a floating point framebuffer on the current generation of cards is pretty difficult, because no blending operations are supported, and the primary thing we need to do is add light contributions together in the framebuffer. The workaround is to copy the part of the framebuffer you are going to reference to a texture, and have your fragment program explicitly add that texture, instead of having the separate blend unit do it. This is intrusive enough that I probably won't hack up the current codebase, instead playing around on a forked version.
Floating point framebuffers and complex fragment shaders will also allow much better volumetric effects, like volumetric illumination of fogged areas with shadows and additive/subtractive eddy currents.
John Carmack
-----------------------------------------
John Carmack's .plan for Feb 07, 2003
-----------------------------------------
The machinima music video that Fountainhead Entertainment (my wife's company) produced with Quake based tools is available for viewing and voting on at: http://www.mtv.com/music/viewers_pick/ ("In the waiting line")
I thought they did an excellent job of catering to the strengths of the medium, and not attempting to make a game engine compete (poorly) as a general purpose renderer. In watching the video, I did beat myself up a bit over the visible popping artifacts on the environment mapping, which are a direct result of the normal vector quantization in the md3 format. While it isn't the same issue (normals are full floating point already in Doom), it was the final factor that pushed me to do the per-pixel environment mapping for the new cards in the current engine.
The neat thing about the machinima aspect of the video is that they also have a little game you can play with the same media assets used to create the video. Not sure when it will be made available publicly.
-----------------------------------------
John Carmack's .plan for Dec 31, 2004
-----------------------------------------
Welcome
I get a pretty steady trickle of emails from people hoping for .plan file updates. There were two main factors involved in my not doing updates for a long time - a good chunk of my time and interest was sucked into Armadillo Aerospace, and the fact that the work I had been doing at Id for the last half of Doom 3 development was basically pretty damn boring.
The Armadillo work has been very rewarding from a learning-lots-of-new-stuff perspective, and I'm still committed to the vehicle development, even post X-Prize, but the work at Id is back to a high level of interest now that we are working on a new game with new technology. I keep running across topics that are interesting to talk about, and the Armadillo updates have been a pretty good way for me to organize my thoughts, so I'm going to give it a more general try here. .plan files were appropriate ten years ago, and sort of retro-cute several years ago, but I'll be sensible and use the web.
I'm not quite sure what the tone is going to be - there will probably be some general interest stuff, but a bunch of things will only be of interest to hardcore graphics geeks.
I have had some hesitation about doing this because there are a hundred times as many people interested in listening to me talk about games / graphics / computers as there are people interested in rocket fabrication, and my mailbox is already rather time consuming to get through.
If you really, really want to email me, add a "[JC]" in the subject header so the mail gets filtered to a mailbox that isn't clogged with spam. I can't respond to most of the email I get, but I do read everything that doesn't immediately scan as spam. Unfortunately, the probability of getting an answer from me doesn't have a lot of correlation with the quality of the question, because what I am doing at the instant I read it is more dominant, and there is even a negative correlation for "deep" questions that I don't want to make an off-the-cuff response to.
Quake 3 Source
I intended to release the Q3 source under the GPL by the end of 2004, but we had another large technology licensing deal go through, and it would be poor form to make the source public a few months after a company paid hundreds of thousands of dollars for full rights to it. True, being public under the GPL isn't the same as having a royalty free license without the need to disclose the source, but I'm pretty sure there would be some hard feelings.
Previous source code releases were held up until the last commercial license of the technology shipped, but with the evolving nature of game engines today, it is a lot less clear. There are still bits of early Quake code in Half Life 2, and the remaining licensees of Q3 technology intend to continue their internal developments along similar lines, so there probably won't be nearly as sharp a cutoff as before. I am still committed to making as much source public as I can, and I won't wait until the titles from the latest deal have actually shipped, but it is still going to be a little while before I feel comfortable doing the release.
Random Graphics Thoughts
Years ago, when I first heard about the inclusion of derivative instructions in fragment programs, I couldn't think of anything off hand that I wanted them for. As I start working on a new generation of rendering code, uses for them come up a lot more often than I expected.
I can't actually use them in our production code because it is an Nvidia-only feature at the moment, but it is convenient to do experimental code with the nv_fragment_program extension before figuring out various ways to build funny texture mip maps so that the built in texture filtering hardware calculates a value somewhat like the derivative I wanted.
If you are basically just looking for plane information, as you would for modifying things with texture magnification or stretching shadow buffer filter kernels, the derivatives work out pretty well. However, if you are looking at a derived value, like a normal read from a texture, the results are almost useless because of the way they are calculated. In an ideal world, all of the samples to be differenced would be calculated at once, then the derivatives calculated from there, but the hardware only calculates 2x2 blocks at a time. Each of the four pixels in the block is given the same derivative, and there is no influence from neighboring pixels. This gives derivative information that is basically half the resolution of the screen and sort of point sampled. You can often see this effect with bump mapped environment mapping into a mip-mapped cube map, where the texture LOD changes discretely along the 2x2 blocks. Explicitly coloring based on the derivatives of a normal map really shows how nasty the calculated value is.
Speaking of bump mapped environment sampling.. I spent a little while tracking down a highlight that I thought was misplaced. In retrospect it is obvious, but I never considered the artifact before: With a bump mapped surface, some of the on-screen normals will actually be facing away from the viewer. This causes minor problems with lighting, but when you are making a reflection vector from it, the vector starts reflecting into the opposite hemisphere, resulting in some sky-looking pixels near bottom edges on the model. Clamping the surface normal to not face away isn't a good solution, because you get areas that "see right through" to the environment map, because a reflection past a clamped perpendicular vector doesn't change the viewing vector. I could probably ramp things based on the geometric normal somewhat, and possibly pre-calculate some data into the normal maps, but I decided it wasn't a significant enough issue to be worth any more development effort or speed hit.
Speaking of cube maps.. The edge filtering on cube maps is showing up as an issue for some algorithms. The hardware basically picks a face, then treats it just like a 2D texture. This is fine in the middle of the texture, but at the edges (which are a larger and larger fraction as size decreases) the filter kernel just clamps instead of being able to sample the neighbors in an adjacent cube face. This is generally a non-issue for classic environment mapping, but when you start using cube map lookups with explicit LOD bias inputs (say, to simulate variable specular powers into an environment map) you can wind up with a surface covered with six constant color patches instead of the smoothly filtered coloration you want. The classic solution would be to implement border texels, but that is pretty nasty for the hardware and API, and would require either the application or the driver to actually copy the border texels from all the other faces. Last I heard, upcoming hardware was going to start actually fetching from the other side textures directly. A second-tier chip company claimed to do this correctly a while ago, but I never actually tested it.
Topics continue to chain together, I'll probably write some more next week.
-----------------------------------------
John Carmack's .plan for May 27, 2005
-----------------------------------------
Cell phone adventures
I'm not a cell phone guy. I resisted getting one at all for years, and even now I rarely carry it. To a first approximation, I don't really like talking to most people, so I don't go out of my way to enable people to call me. However, a little while ago I misplaced the old phone I usually take to Armadillo, and my wife picked up a more modern one for me. It had a nice color screen and a bunch of bad java game demos on it. The bad java games did it.
I am a big proponent of temporarily changing programming scope every once in a while to reset some assumptions and habits. After Quake 3, I spent some time writing driver code for the Utah-GLX project to give myself more empathy for the various hardware vendors and get back to some low-level register programming. This time, I decided I was going to work on a cell phone game.
I wrote a couple java programs several years ago, and I was left with a generally favorable impression of the language. I dug out my old "java in a nutshell" and started browsing around on the web for information on programming for cell phones. After working my way through the alphabet soup of J2ME, CLDC, and MIDP, I've found that writing for the platform is pretty easy.
In fact, I think it would be an interesting environment for beginning programmers to learn on. I started programming on an Apple II a long time ago, when you could just do an "hgr" and start drawing to the screen, which was rewarding. For years, I've had misgivings about people learning programming on Win32 (unix / X would be even worse), where it takes a lot of arcane crap just to get to the point of drawing something on the screen and responding to input. I assume most beginners wind up with a lot of block copied code that they don't really understand.
All the documentation and tools needed are free off the web, and there is an inherent neatness to being able to put the program on your phone and walk away from the computer. I wound up using the latest release of NetBeans with the mobility module, which works pretty well. It certainly isn't MSDev, but for a free IDE it seems very capable. On the downside, MIDP debugging sessions are very flaky, and there is something deeply wrong when text editing on a 3.6 ghz processor is anything but instantaneous.
I spent a while thinking about what would actually make a good game for the platform, which is a very different design space than PCs or consoles. The program and data sizes are tiny, under 200k for java jar files. A single texture is larger than that in our mainstream games. The data sizes to screen ratios are also far out of the range we are used to. A 128x128x16+ bit color screen can display some very nice graphics, but you could only store a half dozen uncompressed screens in your entire size budget. Contrast with PCs, which may be up to a few megabytes of display data, but the total game data may be five hundred times that.
You aren't going to be able to make an immersive experience on a 2" screen, no matter what the graphics look like. Moody and atmospheric are pretty much out. Stylish and fun is about the best you can do.
The standard cell phone style discrete button direction pad with a center action button is a good interface for one handed navigation and selection, but it sucks for games, where you really want a game boy style rocking direction pad for one thumb, and a couple separate action buttons for the other thumb. These styles of input are in conflict with each other, so it may never get any better. The majority of traditional action games just don't work well with cell phone style input.
Network packet latency is bad, and not expected to be improving in the foreseeable future, so multiplayer action games are pretty much out (but see below).
I have a small list of games that I think would work out well, but what I decided to work on is DoomRPG - sort of Bard's Tale meets Doom. Step based smooth sliding/turning tile movement and combat works out well for the phone input buttons, and exploring a 3D world through the cell phone window is pretty neat. We talked to Jamdat about the business side of things, and hired Fountainhead Entertainment to turn my proof-of-concept demo and game plans into a full-featured game.
So, for the past month or so I have been spending about a day a week on cell phone development. Somewhat to my surprise, there is very little internal conflict switching off from the high end work during the day with gigs of data and multi-hundred instruction fragment shaders down to texture mapping in java at night with one table lookup per pixel and 100k of graphics. It's all just programming and design work.
It turns out that I'm a lot less fond of Java for resource-constrained work. I remember all the little gripes I had with the Java language, like no unsigned bytes, and the consequences of strong typing, like no memset, and the inability to read resources into anything but a char array, but the frustrating issues are details down close to the hardware.
The biggest problem is that Java is really slow. On a pure cpu / memory / display / communications level, most modern cell phones should be considerably better gaming platforms than a Game Boy Advanced. With Java, on most phones you are left with about the CPU power of an original 4.77 mhz IBM PC, and lousy control over everything.
I spent a fair amount of time looking at java byte code disassembly while optimizing my little rendering engine. This is interesting fun like any other optimization problem, but it alternates with a bleak knowledge that even the most inspired java code is going to be a fraction the performance of pedestrian native C code.
Even compiled to completely native code, Java semantic requirements like range checking on every array access hobble it. One of the phones (Motorola i730) has an option that does some load time compiling to improve performance, which does help a lot, but you have no idea what it is doing, and innocuous code changes can cause the compilable heuristic to fail.
Write-once-run-anywhere. Ha. Hahahahaha. We are only testing on four platforms right now, and not a single pair has the exact same quirks. All the commercial games are tweaked and compiled individually for each (often 100+) platform. Portability is not a justification for the awful performance.
Security on a cell phone is justification for doing something, but an interpreter isn't a requirement - memory management units can do just as well. I suspect this did have something to do with Java's adoption early on. A simple embedded processor with no MMU could run arbitrary programs securely with java, which might make it the only practical option. However, once you start using blazingly fast processors to improve the awful performance, a MMU with a classic OS model looks a whole lot better.
Even saddled with very low computing performance, tighter implementation of the platform interface could help out a lot. I'm not seeing very conscientious work on the platforms so far. For instance, there is just no excuse for having 10+ millisecond granularity in timing. Given that the java paradigm is sort of thread-happy anyway, having a real scheduler that Does The Right Thing with priorities and hardware interfacing would be an obvious thing. Pressing a key should generate a hardware interrupt, which should immediately activate the key listening thread, which should be able to immediately kill an in-process rendering and restart another one if desired. The attitude seems to be 15 msec here, 20 there, stick it on a queue, finish up a timeslice, who cares, right?
I suspect I will enjoy working with BREW, the competing standard for cell phone games. It lets you use raw C/C++ code, or even, I suppose, assembly language, which completely changes the design options. Unfortunately, they only have a quarter the market share that the J2ME phones have. Also, the relatively open java platform development strategy is what got me into this in the first place - one night I just tried writing a program for my cell phone, which isn't possible for the more proprietary BREW platform.
I have a serious suggestion for the handset designers to go with my idle bitching. I have been told that fixing data packet latency is apparently not in the cards, and it isn't even expected to improve much with the change to 3G infrastructure. Packet data communication seems more modern, and has the luster of the web, but it is worth realizing that for network games and many other flashy Internet technologies like streaming audio and video, we use packets to rather inefficiently simulate a switched circuit.
Cell phones already have a very low latency digital data path - the circuit switched channel used for voice. Some phones have included cellular modems that use either the CSD standard (circuit switched data) at 9.8Kbits or 14.4Kbits or the HSCSD standard (high speed circuit switched data) at 38.4Kbits or 57.6Kbits. Even the 9.8Kbit speed would be great for networked games. A wide variety of two player peer-to-peer games and multiplayer packet server based games could be implemented over this with excellent performance. Gamers generally have poor memories of playing over even the highest speed analog modems, but most of the problems are due to having far too many buffers and abstractions between the data producers/consumers and the actual wire interface. If you wrote eight bytes to the device and it went in the next damned frame (instead of the OS buffer, which feeds into a serial FIFO, which goes into another serial FIFO, which goes into a data compressor, which goes into an error corrector, and probably a few other things before getting into a wire frame), life would be quite good. If you had a real time scheduler, a single frame buffer would be sufficient, but since that isn't likely to happen, having an OS buffer with accurate queries of the FIFO positions is probably best. The worst gaming experiences with modems weren't due to bandwidth or latency, but to buffer pileup.
-----------------------------------------
John Carmack's .plan for May 02, 2006
-----------------------------------------
Orcs & Elves
I'm not managing to make regular updates here, but I'll keep this around just in case. I have a bunch of things that I want to talk about -- some thoughts on programming style and reliability, OpenGL, Xbox 360, etc, but we have a timely topic with the release of our second mobile game, Orcs & Elves, that has spurred me into making this update.
DoomRPG, our (Id Software's and Fountainhead Entertainment's) first mobile title, has been very successful, both in sales and in awards. I predict that the interpolated turn based style of 3D gaming will be widely adopted on the mobile platform, because it plays very naturally on a conventional cell phone. Gaming will be a lot better when there is a mass market of phones that can be played more like a gamepad, but you need to make do with what you actually have.
One of the interesting things about mobile games is that the sales curve is not at all like the drastically front loaded curve of a PC or console game. DoomRPG is selling better now than when it was initially released, and the numbers are promising for supporting additional development work. However, unless I am pleasantly surprised, the hardware capabilities are going to advance much faster than the market in the next couple years, leading to an unusual situation where you can only afford to develop fairly crude games on incredibly powerful hardware. Perhaps "elegantly simple" would be the better way of looking at it, but it will still wind up being like developing an Xbox title for $500,000. That will wind up being great for many small game companies that just want to explore an idea, but having resource far in excess of your demands does minimize the value of being a hot shot programmer. :-)
To some degree this is already the case on high end BREW phones today. I have a pretty clear idea what a maxed out software renderer would look like for that class of phones, and it wouldn't be the PlayStation-esq 3D graphics that seems to be the standard direction. When I was doing the graphics engine upgrades for BREW, I started along those lines, but after putting in a couple days at it I realized that I just couldn't afford to spend the time to finish the work. "A clear vision" doesn't mean I can necessarily implement it in a very small integral number of days. I wound up going with a less efficient and less flexible approach that was simple and robust enough to not likely need any more support from me after I handed it over (it didn't).
During the development of DoomRPG, I had commented that it seemed obvious that it should be followed up with a "traditional, Orcs&Elves sort of fantasy game". A couple people independently commented that "Orcs&Elves" wasn't a bad name for a game so since we didn't run into any obstacles, Orcs& Elves it was. Naming new projects is a lot harder than most people think, because of trademark issues.
In hindsight, we made a strategic mistake at the start of O&E development. We were fresh off the high end BREW version of DoomRPG, and we all liked developing on BREW a lot better than Java. It isn't that BREW is inherently brilliant, it just avoids the deep sucking nature of java for resource constrained platforms (however, note the above about many mobile games not being resource constrained in the future), and allows you to work inside visual studio. O&E development was started high-end first with the low-end versions done afterwards. I should have known better (Anna was certainly suspicious), because it is always easier to add flashy features without introducing any negatives than it is to chop things out without damaging the core value of a game. The high end version is really wonderful, with all the graphics, sound, and gameplay we aimed for, but when we went to do the low end versions, we found that even after cutting the media as we planned, we were still a long way over the 280k java application limit. Rather than just butchering it, we went for pain, suffering, and schedule slippage, eventually delivering a game that still maintained high quality after the de-scoping (the low end platforms still represent the majority of the market). It would have been much easier to go the other way, but the high end phone users will be happy with our mistake.
DoomRPG had three base platforms that were customized for different phones -- Java, low end BREW, and high end BREW. O&E added a high end java version that kept most of the quality of the high end BREW version on phones fast enough to support it from carriers willing to allow the larger download. The download size limits are probably the most significant restriction for gaming on the high end phones. I don't really understand why the carriers encourage streaming video traffic, but balk at a couple megs of game media.
I am really looking forward to the response to Orcs&Elves, because I think it is one of the best product evolutions I have been involved in. The core game play mechanics that were laid out in DoomRPG have proven strong and versatile (again, I bet we have a stable genre here), but now we have a big bag of tricks and a year of polishing the experience behind us, along with a world of some depth. I found it a very good indicator that play testers almost always lost track of time while playing.
This project was doubly nostalgic for me -- the technology was over a decade old for me, but the content took me back twenty years. All the computer games I wrote in high school were adventure games, and my first two commercial sales were Ultima style games for the Apple II, but Id Software never got around to doing one. Old timers may recall that we were going to do a fantasy game called "The Fight For Justice" (starring a hero called Quake...) after Commander Keen, but Wolfenstein 3D and the birth of the FPS sort of got in the way. :-)
-----------------------------------------
John Carmack's .plan for Nov 02, 2007
-----------------------------------------
Technology and Games
Source: http://blogs.ign.com/OrcsandElves/2007/11/02/70591/
Most of my time lately is spent working on Rage, id Software's Id Tech 5 based game that runs on PCs, Macs, 360s, and PS3s. A modern high-end game is a really breathtaking consumer of computation and storage resources for anyone that has been around computers for any length of time. Our target platforms have at least 512 mb of ram, almost 20 gb of media storage, and many tens of gflops of computation, but the development environment involves an even more massive deployment, with a terabyte of raw data being generated before the final culling and compression is done. It is easy to be a little nonchalant about the continuous pace of improvement with computing, but I still take the time to feel a sense of awe about it all.
I started programming on a Commodore VIC-20 with 4k of ram and a tape drive, and I remember writing absurdly long lines of basic code to save the couple bytes that a new line number would consume. As the years went by, and my projects moved from the Apple II to the PC and the early consoles, I continued to gain new insights and perspective on different problems, and I often thought that it would be fun to go back to one of the early small systems. Now that I "knew what I was doing", I could do a lot more within the tight constraints than I was ever able to before. I actually carried a bunch of old systems around with me from house to house for many years before I reached the conclusion that I never was going to spend any real time on them again, and finally tossed them out.
As technology continued to rapidly advance, I saw a lot of good programmers sort of peel off from high end game development, and migrate to other platforms where their existing skill sets were still exactly what was needed. There was a large contingent of hard core assembly language programmers that never wanted to "get soft" with the move to C for more complex game development, and many of them moved from PCs to platforms like the super Nintendo, and eventually into embedded systems or device firmware. There was another contingent that never wanted to move to windows, and so on.
There is an appeal to working with tighter constraints. The more limited the platform, the closer you can feel you are getting to an "optimal" solution. On a modern big system, there are dozens of ways to accomplish any given task, and it just isn't possible to evaluate all the tradeoffs between the implementations of hundreds of different tasks. On a little system, you have to constrain your design to have a much smaller total number of tasks, and the available options are a lot more reduced. Seeing if something is The Right Thing is a lot easier.
I probably had my personal "moment of truth" around the beginning of Doom 3's development, when it became clear that it is no longer possible to deeply understand every single part of a modern application. There is just too much. Nevertheless, I found that I could still enjoy my work when confined to a subset of the entire project, and I have thus remained committed to the high end. However, the appeal of smaller systems still lingers.
A couple years ago, almost on a whim, I got involved in developing games for mobile phones. The primary instigator was when I ran a bunch of games on a new phone and was left frankly appalled at how poor they were. I was thinking to myself "I wrote better games than this each month before we founded Id". I downloaded the SDK for the phone, and started tinkering around a bit. A proof of concept demo and a plan for a game play style specifically tailored for cell phones followed, and we wound up with DoomRPG, and later Orcs & Elves which turned out to be big hits.
In an ideal world, where I could either stop time or clone myself, I would act as a lead programmer for some smaller projects. In the real world, I can't justify spending much time away from the high-end work, so the low-end work gets done in a few short bursts of engine creation and foundation laying, which is then handed over to the Fountainhead team to actually build a great game. After that, Anna mostly uses me as a threat -- if her programmers tell her that something she really wants in a game can't be done, she threatens to call me up and have me tell them how straightforward the problem really is, which usually fuels them to figure out how to do it on their own.
Mobile development was fun in a resource constrained design sort of way, but the programming didn't have the "tight" feel that early gaming had, due to the huge variability in the platforms. When I was messing around on my own phone, I spent some time doing java bytecode disassembly and timing, but it was fairly pointless in light of the two hundred or so different phones the game would wind up running on.
Enter the Nintendo DS.
We had initially looked at possibly moving one of the cell phone titles over to the GBA, but it didn't look like the market would support it, and technically it would have only turned out somewhere in between the low end and high end cell phone versions. With the success of DS, and a suspicion that the players might be a little more open to new third party titles, EA decided that they would support developing a really enhanced version of Orcs&Elves for the DS. This is going a bit out on a limb -- most successful Game Boy / DS titles have been either first-party Nintendo titles, or titles with a strong movie / toy / history tie in. While Orcs & Elves is doing well on mobile, it is still very far from a recognized brand.
The resource limits on the DS make it almost perfect for a small team development. The hardware is fun to work with, you can directly poke at all the registers, the tool chain works well, and the built in limitations of cartridge memory keeps the design decisions fairly straightforward. Going one step farther up to the PSP with a UMD brings you into the realm of large media sizes that can rapidly consume multi-million dollar development budgets.
Once the decision was made to go for it, it was my job to figure out what we could reasonably hope to accomplish on the platform, andbring up a first cut at the 3D rendering engine.
Up next: all the technical details
-----------------------------------------
John Carmack's .plan for Nov 08, 2007
-----------------------------------------
DS Technology
Source: http://blogs.ign.com/OrcsandElves/2007/11/08/71156/
The actual implementation decisions for Orcs&Elves DS were driven by the hardware, the development timeline, and the budget. With a five person team, we had six months to bring it all together on a new platform. I wrote the code for the hardware accelerated 3D renderer and for the remainder of the project, I was technical advisor.
The basic compute power of a 32 bit, 66 mhz arm processor and four megs of ram is a pleasant size to work with, basically about what we had on the PC back when the original Doom was written. You are intrinsically limited to a design that is compact enough that you can wrap your head around every aspect of it at once, but you don't wind up mired in crazy size decisions like trading a couple functions of code for an extra graphical icon.
Going back to fixed point math is always a chore, and while the DS has hardware acceleration for fixed point math, it doesn't automatically integrate with C/C++ code. The compiler / linker / debugger tool chain worked just fine, and I never felt that I was fighting the development environment like it used to be with the really early consoles. The DS SDK takes my preferred approach of both documenting the hardware fully and providing a helper library with full source code that you can pull apart as necessary.
The baseline spec we started with was a 16 meg cart for the game. I was sure we could get all the features we were discussing to fit in there, but in hindsight, we really should have pushed for a 32 meg cart, because it would have allowed us to add a lot more high quality 2D art to the game, and include some pre-rendered cinematics to help set the mood and tell the story. Anna had pushed for this from the beginning, but I was worried that we wouldn't have enough time to create the additional media, and I didn't want to eat the extra manufacturing costs on a speculative game release of an unknown IP.
At first glance, a 16 meg cart is over eight times as large as our high end cell phone distribution, but that turns out to be misleading. Everything is highly compressed on the mobile versions, but because of the need to directly DMA many assets in a usable form on the DS, and sometimes due to the tighter ram limits, a lot of the media takes up more space on the DS. The game is still a lot bigger, but not 8x.
Interfacing with the DS 3D graphics processor was my major contribution to the project. The DS is an unusual graphics machine, with no direct analog before it, but the individual pieces were close enough to things I had experience with that I was able to be effective pretty quickly. While there are a few things I wish would have been done differently, I still found it a lot of fun to work with.
The geometry engine is nice and clean, implementing a good subset of the basic OpenGL pipe in fixed point. It is also quite fast relative to the rest of the system. I had originally laid out the code to double buffer all the command traffic so that the geometry engine could completely overlap with the CPU, but it turned out that we would run into the polygon limits on the rasterizer before the geometry engine worked up much of a sweat, so I just saved memory and let the geometry engine run with a single command buffer. The game logic tended to take more time to process than the geometry engine, so it was almost never a gating factor. A title that heavily used vertex lighting and matrix stack operations might be able to load up the geometry pipeline a bit, but with just texture and color at the verts, it has margin.
Unlike classic OpenGL, you can change the matrix in the middle of specifying a primitive, allowing single bone skinned model rendering to be performed with a conventional vertex pipeline. This was a novel approach that I hadn't seen anywhere else. It was a little odd to see an explicitly separate model and projection matrix in hardware, since they are usually pre-multiplied into a single MVP matrix, but it allows the slow main cpu to avoid having to do some matrix math.
The question of 3D models versus sprites was one of the most significant decisions for Orcs&Elves. There are several situations in the game where you may be facing six or eight monsters, and I was concerned about how low poly the enemies would have to be to avoid problems with the rasterizer, especially when you consider that monsters can chase you into scenes that may tax the rasterizer all by themselves. Coupled with the fact that developing a new skeletal animation system, building all new models, and animating them would almost certainly have busted our development timeline, in the given circumstances, we decided sprites would do a better job.
The rasterization side of things on the DS is... quirky. Almost all 3D rendering systems today use a frame buffer and a depth buffer that is stored in dedicated graphics memory and incrementally updated by the graphics processor. The DS essentially renders as it is being displayed, with only a small set of lines acting as a buffer. This saves memory, and can give speed and power savings, but it has some significant tradeoffs.
Failure modes are bad -- If you overload the polygon list, the remaining polygons just don't show up. This can be mediated by drawing the nearby and important things first, so if anything disappears it is hopefully in the distance where it isn't very noticeable. If you overload the fill rate, horizontal bars appear across the screen. In a perspective view, the greatest complexity and density tend to be in the middle of the screen, so if your scene gets too bad, a jittering color bar tends to appear in the center.
In a conventional rendering engine, overloading complexity just tends to make the game go slower. On the DS, overloading looks broken. This means that you need to leave yourself a lot more margin, and therefore use the hardware less aggressively. The plus side is that since the hardware is re-drawing the screen at 60hz no matter what you do, you are strongly encouraged to make sure the rest of your game also stays at 60hz.
The lack of texture filtering on the DS is the most obvious difference with other current platforms, and it does strongly encourages a cartoony art style for games. The art style for O&E isn't really ideal, and perhaps we should have stylized things a bit more.
You don't have a lot of texture memory available on the DS, less even than an original Playstation. In O&E, the environment graphics (walls and floors) are allocated statically at map load time, and must be balanced by the level artist. The character sprites are dynamically managed, with high resolution artwork for the monster directly in front of the player being swapped in every frame. The field of view and sprite positioning are carefully managed, along with a "pushback factor", to ensure that the sprites are at 1:1 scale when they are one tile away, and 2:1 scale when they are right in front of the player. Without bilinear filtering on the texturing, non-integral scales wind up looking very ugly. This is one of the advantages of the tile based play -- if it was completely free form, the monsters would look a lot uglier. There isn't any intermediate step to be taken to improve the monster rendering without using a full 4x the memory to render an adjacent monster at a 1:1 scale. Even if we had more memory, I probably would have spent it on more animations instead of higher resolution.
The most disappointing mistake in the DS hardware is the lack of even the basic original blending modes from OpenGL. This was a mistake that was common in the very first generation of PC 3D accelerators, where so many companies just assumed that "blending" meant "alpha blending", and they didn't include support for add, modulate, and the other standard blending modes. No relevant company had made that mistake in a decade, and Nintendo's consoles have always done it right since the U64, so it was a surprise to see it in the DS. Additive blending in particular is crucial to most "3D flash" sorts of rendering, and various non-additive modulation modes are used for light mapping and other core rendering features.
I only got to spend four days actually writing all the 3D code for Orcs&Elves, so there are lots of potential directions that I am interested in exploring in the future. We plan to have two more DS projects in development next year, which I hope will let me try out a skeletal animation system, experiment with the networking hardware, and implement a more flexible high level culling algorithm than what I used in O&E.
John Carmack
-----------------------------------------
John Carmack's .plan for Mar 26, 2009
-----------------------------------------
iPhone development: Wolfenstein 3D Classic
Source: http://www.idsoftware.com/wolfenstein3dclassic/wolfdevelopment.htm
I had been frustrated for over a year with the fact that we didn't have any iPhone development projects going internally at Id. I love my iPhone, and I think the App Store is an extremely important model for the software business. Unfortunately, things have conspired against us being out early on the platform.
Robert Duffy and I spent a week early on starting to bring up the Orcs & Elves DS codebase on the iPhone, which would have been a nice project for a launch title, but it wasn't going to be a slam dunk. The iPhone graphics hardware is a more capable superset of the DS hardware (the driver overhead is far, far worse, though), but the codebase was fairly DS specific, with lots of Nintendo API calls all over the place. I got the basics drawing by converting things to OpenGL ES, but I was still on the fence as to whether the best approach to get all the picky little special effects working would be a complete GL conversion, or a DS graphics library emulation layer. Coupled with the fact that the entire user interface would need to be re-thought and re-tested, it was clear that the project would take several months of development time, and need artists and designers as well as coding work. I made the pitch that this would still be a good plan, but the idMobile team was already committed to the Wolfenstein RPG project for conventional Java and BREW mobile phones, and Anna didn't want to slip a scheduled milestone on the established, successful development directions there for a speculative iPhone project.
After thinking about the platform's capabilities a bit more, I had a plan for an aggressive, iPhone specific project that we actually started putting some internal resources on, but the programmer tasked with it didn't work out and was let go. In an odd coincidence, an outside development team came to us with a proposal for a similar project on the Wii, and we decided to have them work on the iPhone project with us instead. We should be announcing this project soon, and it is cool. It is also late, but that's software development...
Late last year, the mobile team had finished up all the planned versions of Wolfenstein RPG, but EA had suggested that in addition to the hundreds of customized versions they normally produce for all the various mobile phones, they were interested in having another team do a significant media quality improvement on it for the iPhone. While Wolf RPG is a very finely crafted product for traditional cell phones, it wasn't designed for the iPhone's interface or capabilities, so it wouldn't be an ideal project, but it should still be worth doing. When we got the first build to test, I was pleased with how the high res artwork looked, but I was appalled at how slow it ran. It felt like one of the mid range java versions, not better than the high end BREW as I expected. I started to get a sinking feeling. I searched around in the level for a view that would confirm my suspicion, and when I found a clear enough view of some angled geometry I saw the tell-tale mid-polygon affine swim in the texture as I rotated. They were using the software rasterizer on the iPhone. I patted myself on the back a bit for the fact that the combination of my updated mobile renderer, the intelligent level design / restricted movement, and the hi-res artwork made the software renderer almost visually indistinguishable from a hardware renderer, but I was very unhappy about the implementation.
I told EA that we were NOT going to ship that as the first Id Software product on the iPhone. Using the iPhone's hardware 3D acceleration was a requirement, and it should be easy -- when I did the second generation mobile renderer (written originally in java) it was layered on top of a class I named TinyGL that did the transform / clip / rasterize operations fairly close to OpenGL semantics, but in fixed point and with both horizontal and vertical rasterization options for perspective correction. The developers came back and said it would take two months and exceed their budget.
Rather than having a big confrontation over the issue, I told them to just send the project to me and I would do it myself. Cass Everitt had been doing some personal work on the iPhone, so he helped me get everything set up for local iPhone development here, which is a lot more tortuous than you would expect from an Apple product. As usual, my off the cuff estimate of "Two days!" was optimistic, but I did get it done in four, and the game is definitely more pleasant at 8x the frame rate.
And I had fun doing it.
Since we now were doing something resembling "real work" on the iPhone at the office, we kept it going at a low priority. One of the projects Cass was tinkering around with at home was a port of Quake 3, and we talked about different interface strategies every now and then.
Unfortunately, when we sat down to try a few things out, we found that Q3 wasn't really running fast enough to make good judgments on iPhone control systems. The hardware should be capable enough, but it will take some architectural changes to the rendering code to get the most out of it.
I was just starting to set up a framework to significantly revise Q3 when I considered the possibility of just going to an earlier codebase to experiment with initially. If we wanted to factor performance out of the equation, we could go all the way back to Wolfenstein 3D, the grandfather of FPS games. It had the basic run and gun play that has been built on for fifteen years, but it originally ran on 286 computers, so it should be pretty trivial to hold a good framerate on the iPhone.
Wolfenstein was originally written in Borland C and TASM for DOS, but I had open sourced the code long ago, and there were several projects that had updated the original code to work on OpenGL and modern operating systems. After a little looking around, I found Wolf3D Redux at http://wolf3dredux.sourceforge.net/. One of the development comments about "removal of the gangrenous 16 bit code" made me smile.
It was nice and simple to download, extract data from a commercial copy of Wolfenstein, and start playing on a PC at high resolution. Things weren't as smooth as they should be at first, but two little changes made a huge difference -- going at VBL synced update rates with one tic per cycle instead of counting milliseconds to match 70 hz game tics, and fixing a bug with premature integralization in the angle update code that caused mouse movement to be notchier than it should be. The game was still fun to play after all these years, and I began to think that it might be worthwhile to actually make a product out of Wolfenstein on the iPhone, rather than just using it as a testbed, assuming the controls worked out as fun to play. The simple episodic nature of the game would make it easy to split up into a $0.99 version with just the first episode, a more expensive version with all sixty levels, and we could release Spear of Destiny if there was additional demand. I was getting a little ahead of myself without a fun-to-play demonstration of feasibility on the iPhone, but the idea of moving the entire line of classic Id titles over -- Wolf, Doom, Quake, Quake 2, and Quake Arena, was starting to sound like a real good idea.
I sent an email to the Wolf 3D Redux project maintainer to see if he might be interested in working on an iPhone project with us, but it had been over a year since the last update, and he must have moved on to other things. I thought about it a bit, and decided that I would go ahead and do the project myself. The "big projects" at Id are always top priority, but the systems programming work in Rage is largely completed, and the team hasn't been gated on me for anything in a while. There is going to be memory and framerate optimization work going on until it ships, but I decided that I could spend a couple weeks away from Rage to work on the iPhone exclusively. Cass continued to help with iPhone system issues, I drafted Eric Will to create the few new art assets, and Christian Antkow did the audio work, but this was the first time I had taken full responsibility for an entire product in a very long time.
@Design notes@
The big question was how "classic" should we leave the game? I have bought various incarnations of Super Mario Bros on at least four Nintendo platforms, so I think there is something to be said for the classics, but there were so many options for improvement. The walls and sprites in the game were originally all 64 x 64 x 8 bit color, and the sound effects were either 8khz / 8 bit mono or (sometimes truly awful) FM synth sounds. Changing these would be trivial from a coding standpoint. In the end, I decided to leave the game media pretty much unchanged, but tweak the game play a little bit, and build a new user framework around the core play experience. This decision was made a lot easier by the fact that we were right around the 10 meg over-the-air app download limit with the converted media. This would probably be the only Id project to ever be within hailing distance of that mark, so we should try to fit it in.
The original in-game status bar display had to go, because the user's thumbs were expected to cover much of that area. We could have gone with just floating stats, but I thought that BJ's face added a lot of personality to the game, so I wanted to leave that in the middle of the screen. Unfortunately, the way the weapon graphics were drawn, especially the knife, caused issues if they were just drawn above the existing face graphics. I had a wider background created for the face, and used the extra space for directional damage indicators, which was a nice improvement in the gameplay. It was a tough decision to stop there on damage feedback, because a lot of little things with view roll kicks, shaped screen blends, and even double vision or blurring effects, are all pretty easy to add and quite effective, but getting farther away from "classic".
I started out with an explicit "open door" button like the original game, but I quickly decided to just make that automatic. Wolf and Doom had explicit "use" buttons, but we did away with them on Quake with contact or proximity activation on everything. Modern games have generally brought explicit activation back by situationally overriding attack, but hunting for push walls in Wolf by shooting every tile wouldn't work out. There were some combat tactics involving explicitly shutting doors that are gone with automatic-use, and some secret push walls are trivially found when you pick up an item in front of them now, but this was definitely the right decision.
You could switch weapons in Wolf, but almost nobody actually did, except for occasionally conserving ammo with the chain gun, or challenges like "beat the game with only the knife". That functionality didn't justify the interface clutter.
The concept of "lives" was still in wolf, with 1-ups and extras at certain scores. We ditched that in Doom, which was actually sort of innovative at the time, since action games on computers and consoles were still very much take-the-quarter arcade oriented. I miss the concept of "score" in a lot of games today, but I think the finite and granular nature of the enemies, tasks, and items in Wolf is better suited to end-of-level stats, so I removed both lives and score, but added persistent awards for par time, 100% kills, 100% secrets, and 100% treasures. The award alone wasn't enough incentive to make treasures relevant, so I turned them into uncapped +1 health crumbs, which makes you always happy to find them.
I increased the pickup radius for items, which avoided the mild frustration of having to sometimes make a couple passes at an item when you are cleaning up a room full of stuff.
I doubled the starting ammo on a fresh level start. If a player just got killed, it isn't good to frustrate them even more with a severe ammo conservation constraint. There was some debate about the right way to handle death: respawn with the level as is (good in that you can keep making progress if you just get one more shot off each time, bad in that weapon pickups are no longer available), respawn just as you entered the level (good -- keep your machinegun / chaingun, bad -- you might have 1 health), or, what I chose, restart the map with basic stats just as if you had started the map from the menu.
There are 60 levels in the original Wolf dataset, and I wanted people to have the freedom to easily jump around between different levels and skills, so there is no enforcement of starting at the beginning. The challenge is to /complete /a level, not /get to/ a level. It is fun to start filling in the grid of level completions and awards, and it often feels better to try a different level after a death. The only exception to the start-anywhere option is that you must find the entrance to the secret levels before you can start a new game there.
In watching the early testers, the biggest issue I saw was people sliding off doors before they opened, and having to maneuver back around to go through. In Wolf, as far as collision detection was concerned, everything was just a 64x64 tile map that was either solid or passable.
Doors changed the tile state when they completed opening or began closing. There was discussion about magnetizing the view angle towards doors, or somehow beveling the areas around the doors, but it turned out to be pretty easy to make the door tiles only have a solid central core against the player, so players would slide into the "notch" with the door until it opened. This made a huge improvement in playability.
There is definitely something to be said for a game that loads in a few seconds, with automatic save of your position when you exit. I did a lot of testing by playing the game, exiting to take notes in the iPhone notepad, then restarting Wolf to resume playing. Not having to skip through animated logos at the start is nice. We got this pretty much by accident with the very small and simple nature of Wolf, but I think it is worth specifically optimizing for in future titles.
The original point of this project was to investigate FPS control schemes for the iPhone, and a lot of testing was done with different schemes and parameters. I was sort of hoping that there would be one "obviously correct" way to control it, but it doesn't turn out to be the case.
For a casual first time player, it is clearly best to have a single forward / back / turn control stick and a fire button.
Tilt control is confusing for first exposure to the game, but I think it does add to the fun factor when you use it. I like the tilt-to-move option, but people that play a lot of driving games on the iPhone seem to like tilt-to-turn, where you are sort of driving BJ through the levels. Tilt needs a decent deadband, and a little bit of filtering is good. I was surprised that the precision on the accelerometer was only a couple degrees, which makes it poorly suited for any direct mapped usage, but it works well enough as a relative speed control.
Serious console gamers tend to take to the "dual stick" control modes easily for movement, but the placement of the fire button is problematic. Using an index finger to fire is effective but uncomfortable. I see many players just move the thumb to fire, using strafe movement for fine tuning aim. It is almost tempting to try to hijack the side volume switch for fire, but the ergonomics aren't quite right, and it would be very un-Apple-like, and wouldn't be available on the iPod touch (plus I couldn't figure out how...).
We tried a tilt-forward to fire to allow you to keep your thumbs on the dual control sticks, but it didn't work out very well. Forward / back tilt has the inherent variable holding angle problem for anything, and a binary transition point is hard for people to hold without continuous feedback. Better visual feedback on the current angle and trip point would help, but we didn't pursue it much. For a game with just, say, a rocket launcher, shake/shove-to-fire might be interesting, but it isn't any good for wolf.
It was critical for the control sticks to be analog, since digital direction pads have proven quite ineffective on touch screens due to progressive lack of registration during play. With an analog stick, the player has continuous visual feedback of the stick position in most cases, so they can self correct. Tuning the deadband and slide off behavior are important.
Level design criteria has advanced a lot since Wolfenstein, but I wasn't going to open up the option of us modifying the levels, even though the start of the first level is painfully bad for a first time player, with the tiny, symmetric rooms for them to get their nose mashed into walls and turned around in. The idea is that you started the game in a prison cell after bashing your guard over the head, but even with the exact same game tools, we would lead the player through the experience much better now. Some of the levels are still great fun to play, and it is interesting to read Tom Hall and John Romero's designer notes in the old hint manuals, but the truth is that some levels were scrubbed out in only a couple hours, unlike the long process of testing and adjustment that goes on today.
It was only after I thought I was basically done with the game that Tim Willits pointed out the elephant in the gameplay room -- for 95% of players, wandering around lost in a maze isn't very much fun.
Implementing an automap was pretty straightforward, and it probably added more to the enjoyment of the game than anything else. Before adding this, I thought that only a truly negligible amount of people would actually finish all 60 levels, but now I think there might be enough people that get through them to justify bringing the Spear of Destiny levels over later.
When I was first thinking about the project I sort of assumed that we wouldn't bother with music, but Wolf3D Redux already had code that converted the old id music format into ogg, so we would up with support at the beginning, and it turned out pretty good. We wound up ripping the red book audio tracks from one of the later commercial Wolf releases and encoding at a different bitrate, but I probably wouldn't have bothered if not for the initial support. It would have been nice to re-record the music with a high quality MIDI synth, but we didn't have the original MIDI source, and Christian said that the conversion back from the id music format to midi was a little spotty, and would take a fair amount of work to get right. I emailed Bobby Prince, the original composer, to see if he had any high quality versions still around, but he didn't get back with me.
The game is definitely simplistic by modern standards, but it still has its moments. Getting the drop on a brown shirt just as he is pulling his pistol from the holster. Making an SS do the "twitchy dance" with your machine gun. Rounding a corner and unloading your weapon on ... a potted plant. Simplistic plays well on the iPhone.
@Programming notes@
Cass and I got the game running on the iPhone very quickly, but I was a little disappointed that various issues around the graphics driver, the input processing, and the process scheduling meant that doing a locked-at-60-hz game on the iPhone wasn't really possible. I hope to take these up with Apple at some point in the future, but it meant that Wolf would be a roughly two tick game. It is only "roughly" because there is no swapinterval support, and the timer scheduling has a lot of variability in it. It doesn't seem to matter all that much, the play is still smooth and fun, but I would have liked to at least contrast it with the perfect limit case.
It turns out that there were a couple issues that required work even at 30hz. For a game like Wolf, any PC that is in use today is essentially infinitely fast, and the Wolf3D Redux code did some things that were convenient but wasteful. That is often exactly the right thing to do, but the iPhone isn't quite as infinitely fast as a desktop PC.
Wolfenstein (and Doom) originally drew the characters as sparse stretched columns of solid pixels (vertical instead of horizontal for efficiency in interleaved planar mode-X VGA), but OpenGL versions need to generate a square texture with transparent pixels. Typically this is then drawn by either alpha blending or alpha testing a big quad that is mostly empty space. You could play through several early levels of Wolf without this being a problem, but in later levels there are often large fields of dozens of items that stack up to enough overdraw to max out the GPU and drop the framerate to 20 fps. The solution is to bound the solid pixels in the texture and only draw that restricted area, which solves the problem with most items, but Wolf has a few different heavily used ceiling lamp textures that have a small lamp at the top and a thin but full width shadow at the bottom. A single bounds doesn't exclude many texels, so I wound up including two bounds, which made them render many times faster.
The other problem was CPU related. Wolf3d Redux used the original ray casting scheme to find out which walls were visible, then called a routine to draw each wall tile with OpenGL calls. The code looked something like this:
[code]DrawWall( int wallNum ) {
char name[128];
texture_t *tex;
sprintf( name, "walls/%d.tga", wallNum );
tex = FindTexture( name );
...
}
texture_t FindTexture( const char *name ) {
int i;
for ( i = 0 ; i < numTextures ; i++ ) {
if ( !strcmp( name, texture[name]->name ) ) {
return texture[name];
}
}
...
} [/code]
I winced when I saw that at the top of the instruments profile, but again, you could play all the early levels that only had twenty or thirty visible tiles at a time without it actually being a problem. However, some later levels with huge open areas could have over a hundred visible tiles, and that led to 20hz again. The solution was a trivial change to something resembling:
[code]DrawWall( int wallNum ) {
texture_t *tex = wallTextures[wallNum];
...
} [/code]
Wolf3D Redux included a utility that extracted the variously packed media from the original games and turned them into cleaner files with modern formats. Unfortunately, an attempt at increasing the quality of the original art assets by using hq2x graphics scaling to turn the 64x64 art into better filtered 128x128 arts was causing lots of sprites to have fringes around them due to incorrect handling of alpha borders. It wasn't possible to fix it up at load time, so I had to do the proper outline-with-color-but-0-alpha operations in a modified version of the extractor. I also decided to do all the format conversion and mip generation there, so there was no significant CPU time spent during texture loading, helping to keep the load time down. I experimented with the PVRTC formats, but while it would have been ok for the walls, unlike with DXT you can't get a lossless alpha mask out of it, so it wouldn't have worked for the sprites. Besides, you really don't want to mess with the carefully chosen pixels in a 64x64 block very much when you scale it larger than the screen on occasion.
I also had to make one last minute hack change to the original media -- the Red Cross organization had asserted their trademark rights over red crosses (sigh) some time after we released the original Wolfenstein 3D game, and all new game releases must not use red crosses on white backgrounds as health symbols. One single, solitary sprite graphic got modified for this release.
User interface code was the first thing I started making other programmers do at Id when I no longer had to write every line of code in a project, because I usually find it tedious and unrewarding. This was such a small project that I went ahead and did it myself, and I learned an interesting little thing. Traditionally, UI code has separate drawing and input processing code, but on a touchscreen device, it often works well to do a combined "immediate mode interface", with code like this:
[code]if ( DrawPicWithTouch( x, y, w, h, name ) ) {
menuState = newState;
} [/code]
Doing that for the floating user gameplay input controls would introduce a frame of response latency, but for menus and such, it works very well.
One of the worst moments during the development was when I was getting ready to hook up the automatic savegame on app exit. There wasn't any savegame code. I went back and grabbed the original 16 bit dos code for load / save game, but when I compiled I found out that the Wolf3d Redux codebase had changed a lot more than just the near / far pointer issues, asm code, and comment blocks. The changes were sensible things, like grouping more variables into structures and defining enums for more things, but it did mean that I wasn't dealing with the commercially tested core that I thought I was. It also meant that I was a lot more concerned about a strange enemy lerping through the world bug I had seen a couple times.
I seriously considered going back to the virgin codebase and reimplementing the OpenGL rendering from scratch. The other thing that bothered me about the Redux codebase was that it was basically a graft of the Wolf3D code into the middle of a gutted Quake 2 codebase. This was cool in some ways, because it gave us a console, cvars, and the system / OpenGL portable framework, and it was clear the original intention was to move towards multiplayer functionality, but it was a lot of bloat. The original wolf code was only a few dozen C files, while the framework around it here was several times that.
Looking through the original code brought back some memories. I stopped signing code files years ago, but the top of WL_MAIN.C made me smile:
[code]/*
=============================================================================
WOLFENSTEIN 3-D
An Id Software production
by John Carmack
=============================================================================
*/ [/code]
It wasn't dated, but that would have been in 1991.
In the end, I decided to stick with the Redux codebase, but I got a lot more free with hacking big chunks of it out. I reimplemented load / save game (fixing the inevitable pointer bugs involved), and by littering asserts throughout the code, I tracked the other problem down to an issue with making a signed comparison against one of the new enum types that compare as unsigned. I'm still not positive if this was the right call, since the codebase is sort of a mess with lots of vestigial code that doesn't really do anything, and I don't have time to clean it all up right now.
Of course, someone else is welcome to do that. The full source code for the commercial app is available on the web site. There was a little thought given to the fact that if I had reverted to the virgin source, the project wouldn't be required to be under the GPL. Wolf and the app store presents a sort of unique situation -- a user can't just compile the code and choose not to pay for the app, because most users aren't registered developers, and the data isn't readily available, but there is actually some level of commercial risk in the fast-moving iPhone development community. It will not be hard to take the code that is already fun to play, pull a bunch of fun things off the net out of various projects people have done with the code over the years, dust off some old map editors, and load up with some modern quality art and sound.
Everyone is perfectly within their rights to go do that, and they can aggressively try to bury the original game if they want. However, I think there is actually a pretty good opportunity for cooperation. If anyone makes a quality product and links to the original Wolf app, we can start having links to "wolf derived" or "wolf related" projects.
That should turn out to be a win for everyone.
I'm going back to Rage for a while, but I do expect Classic Doom to come fairly soon for the iPhone.
-----------------------------------------
John Carmack's .plan for May 27, 2009
-----------------------------------------
iPhone development: Doom Classic Progress Report
Source: http://www.idsoftware.com/iphone-doom-classic-progress/
I have been spending the majority of my time working on iPhone Doom Classic for several weeks now, and the first beta build went out to some external testers a couple days ago. I am moving back on to Rage for a while, but I expect to be able to finish it up for submission to the App Store next month.
Wolfenstein 3D Classic was a quickie project to satisfy my curiosity and test the iPhone waters, but Doom is a more serious effort. In addition to the millions of people with fond memories of the game, there is still an active gaming / development community surrounding the original Doom, and I don't want to disappoint any of them.
One of the things I love about open sourcing the old games is that Doom has been ported to practically everything with a 32 bit processor, from toasters to supercomputers. We hear from a lot of companies that have moved the old games onto various set top boxes and PDAs, and want licenses to sell them. We generally come to some terms in the five figure range for obscure platforms, but it is always with a bit of a sigh. The game runs, and the demo playbacks look good, but there is a distinct lack of actually caring about the game play itself. Making Doom run on a new platform is only a couple days of work. Making it a really good game on a platform that doesn't have a keyboard and mouse or an excess of processing power is an honest development effort.
To my surprise, Christian was able to dig up the original high quality source material for the Doom sounds, so we have 22khz 16 bit sound effects instead of the 11khz 8 bit ones from the original game. It turns out that I can barely tell the difference, which is a sign that we made good choices way back then about catering the sounds to the output device. If we were on the fence for any resource limits, I would have considered sticking with the originals, but the current OpenAL mixer code has errors with 8 bit source buffers, so I would have had to convert to 16 bit at load time anyway, and just referencing the high quality source media actually speeds up the load times.
The music is all stored as mp3, performed on a high quality synthesizer. For Wolf, we used ogg, because that's what was in the Redux codebase that I started with, but I don't have all that CPU performance margin anymore, so it was necessary to use the iPhone's audio decompression hardware through the AudioQueue services. The music is the largest part of the application, but everything else is still well over the 10 meg cellular app transfer limit, so I'm not tempted to try and squeeze it under like we did with Wolfenstein. Maybe being able to get an app over 3G really isn't very important to its success. The fact that people are downloading Myst on the iPhone is heartening -- I have ideas for leveraging our high end idTech-5 content creation pipeline for a future iPhone game, if people will go for a few hundred meg download.
The toughest question was the artwork. Since Wolf was selling well, I had planned on paying contractors to upscale all the Doom graphics to twice the original resolution. When I pulled all the graphics out and tallied it all up, it looked a lot more marginal than I had expected. There were over two thousand individual pieces of art, and it was over ten megatexels in exactly bounded area, let alone atlas fit or power of two inset. The PVRTC compressed formats would work great for the floors and ceilings, which are all nice 64x64 blocks, but it has issues for both the walls and floors.
PVRTC textures must be power of two and, notably, square. If you want a 256 x 512 texture that needs to repeat in both axis, you need to resample it to 512 x 512 to use PVRTC, which means you lose half your compression benefit and get distorted filter kernels and mip level selections. Even worse, Doom had the concept of composited walls, where a surface was generated by adding, say, a switch texture on top of a wall texture. You can't do that sort of operation with PVRTC images. The clean sheet of paper solution to both problems is to design around textures that the hardware likes and use more geometry where you need to tile or combine them, but Doom wasn't well suited to that approach.
Character sprites don't get repeated, so a lot of them can be packed into a nice square 1024 x 1024 texture to minimize waste, but the PVRTC texture formats aren't well suited to sprite graphics. The DXT1 compression format has an exact bit mask for the alpha channel, which is what you want for an application like this. PVRTC treats alpha like any other color channel, so you get coupling between the alpha and color channels that results in partially transparent pixels ringing outside the desired character boundary. It works fine for things like clouds or fireballs, but not so good for character sprites. It looks like it should be possible to get an exact binary mask with the 2 bit PVRTC mode, which could be combined with a 4 bit PVRTC color texture to get a 6 bpp perfectly outlined sprite, but the multitexture performance on the iPhone, even with PVRTC textures, is not fast enough to prevent missing 30 fps when you have a hoard of monsters in front of you.
We started to do some internal samples of up-scaled artwork to use as reference for getting the contractor quotes, and it just wasn't looking all that spectacular. Doubling the art and smoothing out the edges wasn't buying much. There was certainly a lot of room for improvement, since Doom was designed around a 256 color palette with a limited selection of intensity ramps for lighting, but moving there from the starting point would be tricky. If I went to one of our artists today and asked them to draw a bad-ass Baron of Hell throwing a fireball in a 256 x 256 x 16 bit block, I would get something a LOT better than the original art, but it would look different, not just better.
I was also a little taken aback by some of the backlash against the updated graphics that I put in for Wolf 1.1. I took the walls, guns, and decorative sprites from the Mac version of Wolfenstein, and had Eric use that as source to recreate some similar graphics that weren't present in the Mac version. After release, there were a number of reviews that complained, saying that it "ruined the classic feel". I have a couple thoughts abut this: Changing the look on a point release is going to cause some level of complaint, so it is probably a good idea to make any changes from " classic" you think you might want in version 1.0. I also believe most of the complaints were due to the view weapons. The original gun artwork wasn't great, but the double-res ones weren't very good either, and they were a bit different looking. I debated with myself a bit about using them at all, and it looks like I probably shouldn't have. I can't see any drawback whatsoever to the double res walls and sprites, since they are in the same style, just better looking when you jam your face up against them.
In the end, I decided not to do anything with the DOOM source art. With the GPU accelerated filtering and 24 bit lighting it looks a lot better than it ever did, and with floors, ceilings, and lighting you don't seem to notice the low resolution as much as with Wolf.
With the speed (a solid 30 fps, even in the more aggressive later levels), the audio, the resolution, and the rendering quality, it is Doom as you remember it, which is quite a bit better than it actually was. If you go back and play the original game on a 386 with a sound blaster, you will be surprised at the 15 fps, FM-synth music, "bathroom tile sized" 320 x 200 pixels, external network game setup utility, and external keyboard configuration. A lot of people remember it as "The best game EVER!", but "ever" has sure moved a lot in the last decade!
Before I actually started coding on the project, I had visions of adding a lot of modern tuned effects to the core gameplay experience. It would certainly stay a sprite-and-sector based game, but there are many things that would be done differently with the benefit of a GPU and the wisdom of hindsight. Once I began actually working on it, it started to look like a bad idea for a number of reasons. I am trying to not be very disruptive in the main codebase, because I want it to stay a part of http://prboom.sourceforge.net/ instead of being another codebase fork. While I can certainly add a bunch of new features fairly quickly, iterating through a lot of user testing and checking for problems across the >100 commercial Doom levels would take a lot longer. There really is value in " classic" in this case, and there would be some degree of negative backlash to almost any "improvements" I made. There will still be a couple tiny tweaks, but nothing radical is changing in the basic play. It would be fun to take a small team, permanently fork it, and make a "Doom++" just for the iPhone, but that wouldn't be the best first move. Maybe later.
The iPhone interface around the game is all done nicely. Wolf Classic got dinged a bit for the blocky look of the buttons and interface components. I didn't actually see any complaints about the crappy monospace font, but it deserved some. Everything looks good now.
The initial release will be for OS 2.x, and support multiplayer over WiFi. A later release will be for 3.x only, and support bluetooth multiplayer. I looked into the possibility of 3G multiplayer, but the latencies just aren't good enough -- I see 380 or so pings from my phone to local servers. This was interesting, because I have talked to other hardware vendors that claim 3G latencies of half that. I'm not sure if there are issues with the iPhone, issues with AT&T's network in Dallas, or if the vendor was just mistaken about what they were getting. One anecdotal report is that iPhones work better in Japan than here, so it may be infrastructure.
I will probably have another update later with more technical details about the logic behind the new rendering architecture (rewritten for > 2x the speed of the original prBoom GL renderer), touch control issues, and so on.
-----------------------------------------
John Carmack's .plan for Nov 03, 2009
-----------------------------------------
iPhone development: Doom Classic
Source: http://www.idsoftware.com/doom-classic/doomdevelopment.htm
Way back in March when I released the source for Wolfenstein 3D Classic, I said that Doom Classic would be coming "real soon", and on April 27, I gave a progress report: http://www.idsoftware.com/iphone-doom-classic-progress/
I spent a while getting the multiplayer functionality up, and I figured I only had to spend a couple days more to polish things up for release.
However, we were finishing up the big iPhone Doom Resurrection project with Escalation Studios, and we didn't want to have two Doom games released right on top of each other, so I put Doom Classic aside for a while. After Doom Resurrection had its time in the sun, I was prepared to put the rest of the work into Doom Classic, but we ran into another schedule conflict. As I related in my Wolf Classic notes http://www.idsoftware.com/wolfenstein-3d-classic-platinum/wolfdevelopment.htm , Wolfenstein RPG for the iPhone was actually done before Wolfenstein Classic, but EA had decided to sit on it until the release of the big PC / console Wolfenstein game in August.
I really thought I was going to go back and finish things up in September, but I got crushingly busy on other fronts. In an odd little bit of serendipity, after re-immersing myself in the original Doom for the iPhone, I am now working downstairs at Id with the Doom 4 team. I'm calling my time a 50/50 split between Rage and Doom 4, but the stress doesn't divide. September was also the month that Armadillo Aerospace flew the level 2 Lunar Lander Challenge:
http://www.armadilloaerospace.com/n.x/Armadillo/Home/News?news_id=368
Finally, in October I SWORE I would finish it, and we aimed for a Halloween release. We got it submitted in plenty of time, but we ran into a couple approval hiccups that caused it to run to the very last minute. The first was someone incorrectly thinking that the "Demos" button that played back recorded demos from the game, was somehow providing demo content for other commercial products, which is prohibited. The second issue was the use of an iPhone image in the multiplayer button, which we had to make a last minute patch for.
@Release notes@
Ok, the game is finally out (the GPL source code is being packaged up for release today). Based on some review comments, there are a couple clarifications to be made:
Multiplayer requires a WiFi connection that doesn't have UDP port 14666 blocked. I'm quite happy with the simple and fast multiplayer setup, but it seems like many access points just dump the packets in the trash. If the multiplayer button on the main menu doesn't start pulsing for additional players after the first player has hit it, you won't be able to connect. I have also seen a network where the button would pulse, but the player would never get added to the player list, which meant that somehow the DNS packets were getting through, but the app packets weren't. It works fine on a normal AirPort install... More on networking below.
I took out tilt-to-turn just to free up some interface screen space, because I didn't know anyone that liked that mode, and my query thread on Touch Arcade didn't turn up people that would miss it a lot.
Evidently there are a few people that do care a lot, so we will cram that back in on the next update. The functionality is still there without a user interface, so you can enable it by four-finger-tapping to bring up the keyboard and typing "tiltturn 4000" or some number like that, and it will stay set. Make sure you have tiltmove pulled down to 0. I never got around to putting in a real console, but you can change a few parameters like that, as well as enter all the original doom cheat codes like IDDQD, IDKFA, etc.
I think that the auto-centering control sticks in Doom Classic are a better control scheme than the fixed sticks from Wolf Classic. The advice for wolf was to adjust the stick positions so that your thumbs naturally fell in the center point, so I just made that automatic for Doom. Effective control always involved sliding your thumbs on the screen, rather than discretely tapping it, and this mode forces you to do that from the beginning. Still, even if the new mode is some fraction "better", there are a lot of people who have logged a lot of hours in Wolfenstein Classic, and any change at all will be a negative initially. In the options->settings menu screen, there is a button labeled "Center sticks: ON" that can be toggled off to keep the sticks fixed in place like in Wolf.
A subtle difference is that the turning sensitivity is now graded so that a given small movement will result in a specific percentage increase in speed, no matter where in the movement range it is. With linear sensitivity, if you are 10 pixels off from the center and you move your thumb 10 pixels farther, then the speed exactly doubles. If you are 50 pixels off from the center, the same 10 pixel move only increases your turning rate by 20%. With ramped sensitivity, you would get a 20% (depending on the sensitivity scale) increase in speed in both cases, which tends to be better for most people. You can disable this by toggling the "Ramp turn: ON" option off.
In hindsight, I should have had a nice obvious button on the main options screen that said "Wolfenstein Style" and had the same options, but I have always had difficult motivating myself to do good backwards compatibility engineering. Even then, the movement speeds are different between the games, so it wouldn't have felt exactly the same.
It was a lot of fun to do this project, working on it essentially alone, as a contrast to the big teams on the major internal projects. I was still quite pleased with how the look and feel of the game holds up after so long, especially the "base style" levels. The "hell levels" show their age a lot more, where the designers were really reaching beyond what the technology could effectively provide.
@Future iPhone work@
We do read all the reviews in the App store, and we do plan on supporting Doom Classic with updates. Everything is still an experiment for us on the iPhone, and we are learning lessons with each product. At this point, we do not plan on making free lite versions of future products, since we didn't notice anything worth the effort with Wolfenstein, and other developers have reported similar findings.
We have two people at Id that are going to be dedicated to iPhone work. I doubt I will be able to personally open Xcode again for a few months, but I do plan on trying to work out a good touch interface for Quake Classic and the later 6DOF games. I also very much want to make at least a tech demo that can run media created with a version of our idTech 5 megatexture content creation pipeline. I'm not sure exactly what game I would like to do with it, so it might be a 500 mb free gee-whiz app...
Wolfenstein Classic Platinum was a break-in opportunity for the new internal iPhone developers. We were originally planning on making the Spear of Destiny levels available as in-app purchased content. Then we decided to make it a separate "Platinum Edition" application at a reasonable price. Finally, we decided that we would just make it a free update, but something has gone wrong during this process -- people who buy the app for the first time get everything working properly, but many people who upgrade the App from a previous purchase are seeing lots of things horribly broken. We are working with Apple to try to debug and fix this, but the workaround is to uninstall the app completely, then reload it. The exciting thing about Wolf Platinum is the support for downloadable levels, which is the beta test for future game capabilities. Using a URL to specify downloadable content for apps is a very clean way to interface to the game through a web page or email message.
The idMobile team is finishing up the last of the BREW versions of Doom 2 RPG, and work has started on an iPhone specific version, similar to the Wolfenstein RPG release. The real-time FPS games are never going to be enjoyable for a lot of people, and the turn based RPG games are pretty neat in many regards. If they are well received, we will probably bring over the Orcs&Elves games as well.
I want to work on a Rage themed game to coincide with Rage's release, but we don't have a firm direction or team chosen for it. I was very excited about doing a really-designed-for-the-iPhone first person shooter, but at this point I am positive that I don't have the time available for it.
@Networking techie stuff@
I doubt one customer in ten will actually play a network game of Doom Classic, but it was interesting working on it.
Way back in March when I was first starting the work, I didn't want the game to require 3.0 to run, and I generally try to work with the lowest level interfaces possible for performance critical systems, so I wasn't looking at GameKit for multiplayer. I was hoping that it was possible to use BSD sockets to allow both WiFi networking on 2.0 devices and WiFi or ad-hoc bluetooth on 3.0 devices. It turns out that it is possible, but it wasn't documented as such anywhere I could find.
I very much approve of Apple's strategy of layering Obj-C frameworks on top of Unix style C interfaces. Bonjour is a layer over DNS, and GameKit uses sockets internally. The only bit of obscure magic that goes on is that the bluetooth IP interface only comes into existence after you have asked DNS to resolve a service that was reported for it. Given this, there is no getting around using DNS for initial setup.
With WiFi, you could still use your own broadcast packets to do player finding and stay completely within the base sockets interfaces, and this might even make some sense, considering that there appear to be some WiFi access points that will report a DNS service's existence that your app can't actually talk to.
For every platform I have done networking on previously, you could pretty much just assume that you had the loopback interface and an Ethernet interface, and you could just use INADDR_ANY for pretty much everything. Multiple interfaces used to just be an issue for big servers, but the iPhone can have a lot of active interfaces -- loopback, WiFi Ethernet, Bluetooth Ethernet, and several point to point interfaces for the cellular data networks.
At first, I was excited about the possibility of multiplayer over 3G. I had been told by someone at Intel that they were seeing ping times of 180 ms on 3G devices, which could certainly be made to work for gaming.
Unfortunately, my tests, here in Dallas at least, show about twice that, which isn't worth fighting. I'm a bit curious whether they were mistaking one-way times, or if the infrastructure in California is really that much better. In any case, that made my implementation choice clear -- local link networking only.
A historical curiosity: the very first release of the original Doom game on the PC used broadcast IPX packets for LAN networking. This seemed logical, because broadcast packets for a network game of N players has a packet count of just N packets on the network each tic, since everyone hears each packet. The night after we released the game, I was woken up by a call from a college sysadmin yelling at me for crippling their entire network. I didn't have an unlisted number at the time. When I had decided to implement network gaming, I bought and read a few books, but I didn't have any practical experience, so I had thought that large networks were done like the books explained, with routers connecting independent segments. I had no idea that there were many networks with thousands of nodes connected solely by bridges that forwarded all broadcast packets over lower bit rate links. I quickly changed the networking to have each peer send addressed packets to the other peers. More traffic on the local segment, but no chance of doing horrible things to bridged networks.
WiFi is different from wired Ethernet in a few ways. WiFi clients don't actually talk directly to each other, they talk to the access point, which rebroadcasts the packet to the destination, so every packet sent between two WiFi devices is actually at least two packets over the air.
An ad-hoc WiFi network would have twice the available peer to peer bandwidth and half the packet drop rate that an access point based one does. Another point is that unlike wired Ethernet, the WiFi link level actually does packet retransmits if the destination doesn't acknowledge receipt. They won't be retransmitted forever, and the buffer spaces are limited, so it can't be relied upon the way you do TCP, but packet drops are more rare than you would expect. This also means that there are lots of tiny little ACK packets flying around, which contributes to reduced throughput. Broadcast packets are in-between -- the broadcast packet is sent from the source to the access point with acknowledgment and retransmits, but since the access point can't know who it is going to, it just fires it out blind a single time.
I experimentally brought the iPhone networking up initially using UDP broadcast packets, but the delivery was incredibly poor. Very few packets were dropped, but hundreds of milliseconds could sometimes go by with no deliveries, then a huge batch would be delivered all at once. I thought it might be a policy decision on our congested office access point, but it behaved the same way at my house on a quiet LAN, so I suspect there is an iPhone system software issue. If I had a bit more time, I would have done comparisons with a WiFi laptop. I had pretty much planned to use addressed packets anyway, but the broadcast behavior was interesting.
Doom PC was truly peer to peer, and each client transmitted to every other client, for N * (N-1) packets every tic. It also stalled until valid data had arrived from every other player, so adding more players hurts in two different ways -- more packets = more congestion = more likely to drop each individual packet. The plus side of an arrangement like this is that it is truly fair, no client has any advantage over any other, even if one or more players are connected by a lower quality link. Everyone gets the worst common denominator behavior.
I settled on a packet server approach for the iPhone, since someone had to be designated a "server" anyway for DNS discovery, and it has the minimum non-broadcast packet count of 2N packets every tic. Each client sends a command packet to the server each tic, the server combines all of them, then sends an addressed packet back to each client. The remaining question was what the server should do when it hasn't received an updated command from a client. When the server refused to send out a packet until it had received data from all clients, there was a lot more variability in the arrival rate. It could be masked by intentionally adding some latency on each client side, but I found that it plays better to just have the server repeat the last valid command when it hasn't gotten an update. This does mean that there is a slight performance advantage to being the server, because you will never drop an internal packet.
The client always stalls until it receives a server packet, there was no way I had the time to develop any latency reducing / drop mitigating prediction mechanisms. There are a couple full client / server, internet capable versions of Doom available on the PC, but I wanted to work from a more traditional codebase for this project.
So, I had the game playing well over WiFi, but communication over the Bluetooth interface was significantly worse. There was an entire frame of additional latency versus WiFi, and the user mode Bluetooth daemon was also sucking up 10% of the CPU. That would have been livable, but there were regular surges in the packet delivery rate that made it basically unplayable.
Surging implies a buffer somewhere backing up and then draining, and I had seen something similar but less damaging occasionally on WiFi as well, so I wondered if there might be some iPhone system communication going on. I spent a little while with WireShark trying to see if the occasional latency pileup was due to actual congestion, and what was in the packets, but I couldn't get my Macbook into promiscuous WiFi mode, and I didn't have the time to configure a completely new system.
In the end, I decided to just cut out the Bluetooth networking and leave it with WiFi. There was a geek-neatness to having a net game with one client on WiFi and another on Bluetooth, but I wasn't going to have time to wring it all out.
-----------------------------------------
John Carmack's .plan for Oct 26, 2010
-----------------------------------------
RAGE on iPhone/iPad/iPod
Source: http://www.bethblog.com/index.php/2010/10/29/john-carmack-discusses-rage-on-iphoneipadipod-touch/
@RAGE for iPhone@
Our mobile development efforts at id took some twists and turns in the last year. The plan was always to do something RAGE-related on the iPhone/iPad/iPod touch next, but with all the big things going on at id, the mobile efforts werenít front and center on the priority list. There had been a bit of background work going on, but it was only towards the end of July that I was able to sit down and write the core engine code that would drive the project.
I was excited about how well it turned out, and since this was right before QuakeCon, I broke with tradition and did a live technology demo during my keynote. In hindsight, I probably introduced it poorly. I said something like ìIts RAGE. On the iPhone. At 60 frames a second.î Some people took that to mean that the entire PC/console game experience was going to be on the iPhone, which is definitely not the case.
What I showed was a technology demo, written from scratch, but using the RAGE content creation pipeline and media. We do not have the full RAGE game running on iOS, and we do not plan to try. While it would (amazingly!) actually be possible to compile the full-blown PC/console RAGE game for an iPhone4 with some effort, it would be a hopelessly bad idea. Even the latest and greatest mobile devices are still a fraction of the power of a 360 or PS3, let alone a high end gaming PC, so none of the carefully made performance tradeoffs would be appropriate for the platform, to say nothing of the vast differences in controls.
What we do have is something unlike anything ever seen on the iOS platforms. It is glorious, and a lot of fun. Development has been proceeding at high intensity since QuakeCon, and we hope to have the app out by the end of November.
The technical decision to use our megatexture content creation pipeline for the game levels had consequences for its scope. The data required for the game is big. Really, really big. Seeing Myst do well on the iPhone with a 700 meg download gave me some confidence that users would still download huge apps, and that became the target size for our standard definition version, but the high definition version for iPad / iPhone 4 will be around twice that size. This is more like getting a movie than an app, so be prepared for a long download. Still, for perspective, the full scale RAGE game is around 20 gigs of data with JPEG-XR compression, so 0.7 gigs of non-transcoded data is obviously a tiny slice of it.
Since we werenít going to be able to have lots of hugely expansive levels, we knew that there would be some disappointment if we went out at a high price point, no matter how good it looked. We have experimented with a range of price points on the iPhone titles so far, but we had avoided the very low end. We decided that this would be a good opportunity to try a $0.99 SD / $1.99 HD price point. We need to stay focused on not letting the project creep out of control, but I think people will be very happy with the value.
The little slice of RAGE that we decided to build the iPhone product around is ìMutant Bash TVî, a post apocalyptic combat game show in the RAGE wasteland. This is the perfect setup for a quintessential first person shooter game play experience ó you pick your targets, aim your shots, time your reloads, dodge the bad guys, and try and make it through to the end of the level with a better score than last time. Beyond basic survival, there are pickups, head shots, and hit streak multipliers to add more options to the gameplay, and there is a broad range of skill levels available from keep-hitting-fire-and-you-should-make-it to almost-impossible.
A large goal of the project has been to make sure that the levels can be replayed many times. The key is making the gamplay itself the rewarding aspect, rather than story progression, character development, or any kind of surprises. Many of the elements that made Doom Resurrection good the first time you played it hurt the replayability, for instance. RAGE iOS is all action, all the time. I have played the game dozens of times, and testing it is still fun instead of a chore.
@Technical Geek Details@
The id Tech 5 engine uses a uniform paged virtual texture system for basically everything in the game. While the algorithm would be possible on 3GS and later devices, it has a substantial per-fragment processing cost, and updating individual pages in a physical texture is not possible with PVRTC format textures. The approach used for mobile RAGE is to do the texture streaming based on variable sized contiguous ìtexture islandsî in the world. This is much faster, but it forces geometric subdivision of large surfaces, and must be completely predictive instead of feedback reactive. Characters, items, and UI are traditionally textured.
We build the levels and preview them in RAGE on the PC, then run a profiling / extraction tool to generate the map data for the iOS game. This tool takes the path through the game and determines which texture islands are going to be visible, and at what resolution and orientation. The pixels for the texture island are extracted from the big RAGE page file, then anisotropically filtered into as many different versions as needed, and packed into 1024◊1024 textures that are PVRTC compressed for the device.
The packing into the textures has conflicting goals ñ to minimize total app size you want to cram texture islands in everywhere they can fit, but you also donít want to scatter the islands needed for a given view into a hundred different textures, or radically change your working set in nearby views. As with many NP complete problems, I wound up with a greedy value metric optimizing allocation strategy.
Managing over a gig of media made dealing with flash memory IO and process memory management very important, and I did a lot of performance investigations to figure things out.
Critically, almost all of the data is static, and can be freely discarded. iOS does not have a swapfile, so if you use too much dynamic memory, the OS gives you a warning or two, then kills your process. The bane of iOS developers is that ìtoo muchî is not defined, and in fact varies based on what other apps (Safari, Mail, iPod, etc) that are in memory have done. If you read all your game data into memory, the OS canít do anything with it, and you are in danger. However, if all of your data is in a read-only memory mapped file, the OS can throw it out at will. This will cause a game hitch when you need it next, but it beats an abrupt termination. The low memory warning does still cause the frame rate to go to hell for a couple seconds as all the other apps try to discard things, even if the game doesnít do much.
Interestingly, you can only memory map about 700 megs of virtual address space, which is a bit surprising for a 32 bit OS. I expected at least twice that, if not close to 3 gigs. We sometimes have a decent fraction of this mapped.
A page fault to a memory mapped file takes between 1.8 ms on an iPhone 4 and 2.2 ms on an iPod 2, and brings in 32k of data. There appears to be an optimization where if you fault at the very beginning of a file, it brings in 128k instead of 32k, which has implications for file headers.
I am pleased to report that fcntl( fd, F_NOCACHE ) works exactly as desired on iOS ñ I always worry about behavior of historic unix flags on Apple OSs. Using this and page aligned target memory will bypass the file cache and give very repeatable performance ranging from the page fault bandwidth with 32k reads up to 30 mb/s for one meg reads (22 mb/s for the old iPod). This is fractionally faster than straight reads due to the zero copy, but the important point is that it wonít evict any other buffer data that may have better temporal locality. All the world megatexture data is managed with uncached reads, since I know what I need well ahead of time, and there is a clear case for eviction. When you are past a given area, those unique textures wonít be needed again, unlike, say monster animations and audio, which are likely to reappear later.
I pre-touch the relevant world geometry in the uncached read thread after a texture read has completed, but in hindsight I should have bundled the world geometry directly with the textures and also gotten that with uncached reads.
OpenAL appears to have a limit of 1024 sound buffers, which we bumped into. We could dynamically create and destroy the static buffer mappings without too much trouble, but that is a reasonable number for us to stay under.
Another behavior of OpenAL that surprised me was finding (by looking at the disassembly) that it touches every 4k of the buffer on a Play() command. This makes some sense, forcing it to page the entire thing into ram so you donít get broken sound mixing, but it does unpredictably stall the thread issuing the call. I had sort of hoped that they were just eating the page faults in the mixing thread with a decent sized mix ahead buffer, but I presume that they found pathological cases of a dozen sound buffers faulting while the GPU is sucking up all the bus bandwidth or some such. I may yet queue all OpenAL commands to a separate thread, so if it has to page stuff in, the audio will just be slightly delayed instead of hitching the framerate.
I wish I could prioritize the queuing of flash reads ñ game thread CPU faults highest, sound samples medium, and textures lowest. I did find that breaking the big texture reads up into chunks helped with the worst case CPU stalls.
There are two project technical decisions that I fretted over a lot:
Because I knew that the basic rendering technology could be expressed with fixed function rendering, the game is written to OpenGL ES 1.1, and can run on the older MBX GPU platforms. While it is nice to support older platforms, all evidence is that they are a negligible part of the market, and I did give up some optimization and feature opportunities for the decision.
It was sort of fun to dust off the old fixed function puzzle skills. For instance, getting monochrome dynamic lighting on top of the DOT3 normal mapping in a single pass involved sticking the lighting factor in the alpha channel of the texture environment color so it feeds through to the blender, where a GL_SRC_ALPHA, GL_ZERO blend mode effects the modulation on the opaque characters. This sort of fixed function trickery still makes me smile a bit, but it isnít a relevant skill in the modern world of fragment shaders.
The other big one is the codebase lineage.
My personally written set of iPhone code includes the renderer for Wolfenstein RPG, all the iPhone specific code in Wolfenstein Classic and Doom Classic, and a few one-off test applications. At this point, I feel that I have a pretty good idea of what The Right Thing To Do on the platform is, but I donít have a mature expression of that in a full game. There is some decent code in Doom Classic, but it is all C, and I would prefer to do new game development in (restrained) C++.
What we did have was Doom Resurrection, which was developed for us by Escalation Studios, with only a few pointers here and there from me. The play style was a pretty close match (there is much more freedom to look around in RAGE), so it seemed like a sensible thing. This fits with the school of thought that says ìnever throw away the codeî (http://www.joelonsoftware.com/articles/fog0000000069.html ). I take issue with various parts of that, and much of my success over the years has involved wadding things up and throwing it all away, but there is still some wisdom there.
I have a good idea what the codebase would look like if I wrote it from scratch. It would have under 100k of mutable CPU data, there wouldnít be a resource related character string in sight, and it would run at 60 fps on new platforms / 30 fps on old ones. Iím sure I could do it in four months or so (but I am probably wrong). Unfortunately, I canít put four months into an iPhone project. Iím pushing it with two months ñ I have the final big RAGE crunch and forward looking R&D to get back to.
So we built on the Resurrection codebase, which traded expediency for various compromise in code efficiency. It was an interesting experience for me, since almost all the code that I normally deal with has my ìcoding DNAî on it, because the id Software coding standards were basically ìprogram the way John does.î The Escalation programmers come from a completely different background, and the codebase is all STL this, boost that, fill-up-the-property list, dispatch the event, and delegate that.
I had been harboring some suspicions that our big codebases might benefit from the application of some more of the various ìmodernî C++ design patterns, despite seeing other large game codebases suffer under them. I have since recanted that suspicion.
I whine a lot about it (occasionally on twitter), and I sometimes point out various object lessons to the other mobile programmers, but in the end, it works, and it was probably the right decision.